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INTRODUCTION 
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INTRODUCTION 


The  principal  goals  of  this  study  were  to  select  a preferred  timing  subsystem  for  the 
future  integrated  switched  digital  Defense  Communications  System  (DCS)  and  to  quantitatively 
justify  the  selected  subsystem.  In  this  introductory  section,  the  outline  of  the  report  is  discussed 
with  the  primary  objective  of  elaborating  on  the  solution  methodology  that  was  undertaken  to 
provide  the  required  outputs. 

Several  techniques  for  timing  and  synchronization  exist  on  which  to  base  either 
selected  portions  or  the  entire  timing  subsystem  for  the  DCS.  These  techniques  are  master-slave, 
pulse  stuffing,  mutual  synchronization,  independent  clocks  and  time  reference  distribution.  The 
initial  step  in  the  solution  methodology  was  to  thoroughly  analyze  these  techniques.  Section  2.0 
presents  an  overview  of  the  techniques  stressing  similarities  and  differences  between  approaches . 

In  addition,  this  section  provides  the  rationale  for  addressing  some  techniques  in  greater  detail 
than  others.  Section  3.0  deals  with  the  baseline  designs  of  the  synchronous  techniques,  the 
purpose  being  that  there  is  great  design  flexibility  within  each  of  the  three  broad  generic  terms. 

A comparative  analysis  of  techniques  would  not  be  meaningful  if  each  technique  was  not  itself 
specifically  defined. 

As  will  be  noted  in  Section  2.0,  perhaps  the  most  basic  function  of  a timing 
subsystem  is  to  ensure  that  each  nodal  clock  runs  at  or  extremely  close  to  the  same  average 
frequency.  As  a result  of  an  initial  examination,  it  was  concluded  that  a common  average 
frequency  could  be  achieved  by  any  of  the  candidate  techniques.  It  thus  became  necessary  to 
select  a more  detailed  set  of  criteria  on  which  to  base  a comparative  analysis  of  the  candidate 
techniques.  The  discussions  presented  in  Sections  4.0,  5.0  and  6.0  constitute  the  set  of  criteria 
or  test  bed  in  which  each  of  the  candidate  techniques  was  evaluated.  Section  4.0  defines  and 
justifies  a set  of  desirable  characteristics  that  if  provided  by  the  candidate  techniques  would 
increase  the  overall  utility  of  the  subsystem.  Section  5.0  outlines  different  network  configurations 
in  which  the  various  techniques  were  evaluated  to  determine  their  relative  performance.  Several 
configurations  rather  than  one  were  selected  since  the  actual  network  configuration  in  which  the 
s elected  timing  subsystem  will  operate  is  not  firmly  defined.  Each  network  configuration  was 
selected  to  be  realistic  and  to  highlight  potential  weaknesses  or  strengths  in  the  techniques. 
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Section  6.0  discusses  a number  of  scenarios  and  the  resultant  stresses  on  the  timing  subsystem. 

It  was  concluded  that  the  degree  to  which  the  candidates  provided  the  desirable  characteristics 
when  confronted  with  the  defined  set  of  scenarios  was  the  best  available  barometer  of  subsystem 
effectiveness. 

Sections  7.0  and  8.0  detail  the  techniques  used  and  the  results  achieved  in 
determining  the  effectiveness  of  each  technique.  Specifically,  Section  7.0  deals  with  the 
simulation  that  was  generated  to  provide  performance  data  on  each  of  the  synchronous  techniques 
within  the  framework  of  specific  networks.  The  models  used  in  the  simulation  are  discussed  as 
well  as  the  results.  Section  8.0  utilizes  the  results  of  the  simulation  exercise  as  well  as  an 
extensive  analysis  effort  to  evaluate  the  degree  that  each  candidate  technique  provides  the 
desirable  characteristics.  Section  9.0  ranks  these  desirable  characteristics  according  to  their 
importance  or  criticality.  An  appropriate  combination  of  the  importance  of  the  characteristics 
and  the  degree  of  the  characteristics  are  provided  by  each  technique  can  then  be  used  to  rate 
the  techniques. 

Section  10.0  provides  relative  cost  estimates  for  the  techniques  to  serve  as  an 
input  along  with  the  performance  evaluations  in  selecting  the  preferred  timing  subsystem. 

Section  11.0  presents  a discussion  of  alternate  timing  subsystems  which  in  general 
are  hybrid  combinations  of  techniques. 

Section  12.0  presents  conclusions  in  terms  of  both  techniques  and  subsystems. 

Section  13.0  outlines  a suggested  test  program  that  could  provide  valuable  additional 

data. 


Section  14.0,  the  concluding  section,  outlines  an  additional  effort  that  appears 
as  the  appropriate  next  step  in  the  development  of  a future  DCS  timing  subsystem. 
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SECTION  2.0 


OVERVIEW  OF  CANDIDATE  TECHNIQUES 


2.0 


OVERVIEW  OF  CANDIDATE  TECHNIQUES 


The  basic  functions  to  be  provided  by  a network  timing  and  synchronization  sub- 
system are  first  to  ensure  that  each  nodal  clock  runs  at  the  same  average  frequency,  and  second, 
to  ensure  a high  probability  of  the  maintenance  of  bit  count  integrity.  It  is  worthwhile  to  point 
out  that  the  first  function  is  necessary  but  not  sufficient  for  the  provision  of  the  second.  The 
second  function  can  only  be  provided  if  sufficient  buffering  exists  to  compensate  both  for 
differences  in  instantaneous  frequency  between  clocks  and  for  the  variations  in  delay  that  exist 
on  the  links  connecting  the  nodal  clocks.  That  is  to  say,  with  enough  buffering,  any  technique 
that  maintains  the  nodal  clocks  at  the  same  average  frequency  will  also  provide  a high  proba- 
bility of  maintaining  bit  count  integrity.  However,  those  techniques  that  require  less  buffering 
for  a given  bit  slip  rate  are  preferable  to  those  that  require  more  buffering.  Although  not 
completely  defendable  from  a cost  point  of  view  (because  of  the  relatively  low-cost  of  buffers^, 
this  statement  is  supportable  since  it  is  certainly  true  that  as  buffer  size  tends  to  increase, 
especially  on  a series  of  tandem  links,  delay,  which  is  undesirable,  also  increases.  In  examining 
the  techniques,  it  is  apparent  that  those  that  maintain  closer  instantaneous  frequencies  between 
nodal  clocks,  especially  in  the  face  of  different  stresses,  and  those  that  can  cope  with  variable 
delays  on  the  links  will  require  less  buffering;  and  therefore,  be  more  desirable. 

Several  timing  techniques  exist  on  which  a timing  subsystem  can  be  based.  Five 
techniques  were  initially  considered  for  examination.  These  were  master-slave,  mutual  syn- 
chronization, time  reference  distribution,  pulse  stuffing  and  independent  clocks.  As  mentioned 
in  Section  1 .0  the  moot  promising  techniques  have  been  scrutinized  in  order  to  determine  the 
degree  to  which  each  provides  the  desirable  characteristics  listed  in  Section  4.0.  This  task 
was  undertaken  because  it  was  felt  that  each  of  the  techniques  could  ensure  that  the  nodal 
clocks  would  run  at  the  same  average  frequency  under  normal  operating  conditions.  However, 
what  had  to  be  explored  was  first,  how  each  system  would  react  under  stress;  second,  what 
ancillary  benefits  would  be  provided  by  the  techniques  and  third,  the  closeness  of  instantaneous 
clock  frequencies  for  each  technique.  The  remainder  of  this  section  will  provide  an  overview 
oc  each  of  the  techniques,  stressing  the  similarities  and  differences  between  techniques. 
Furthermore,  the  rationale  for  pursuing  some  alternatives  in  greater  depth  than  others  will  be 
provided . 
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The  techniques  that  have  been  considered  can  be  divided  into  two  general  cate- 
gories, those  that  are  unlocked  or  undisciplined  clock  approaches  and  those  that  depend  on 
locked  or  disciplined  clocks.  Pulse  stuffing  and  independent  clocks  belong  in  the  first  categoiy; 
mutual  synchronization,  master-slave  and  time  reference  distribution  belong  in  the  second 
category.  After  an  initial  examination,  it  was  decided  that  as  a category,  the  disciplined 
clock  techniques  held  much  greater  promise  than  the  undisciplined  clock  techniques.  This 
decision,  which  was  based  in  the  case  of  pulse  stuffing  from  a cost  point  of  view  and  in  the  case 
of  independent  clocks  on  both  a cost  point  of  view  and  on  the  necessity  of  accepting  periodic 
bit  slips.  This  is  explained  in  the  paragraphs  below  which  summarize  these  two  techniques. 

Pulse  Stuffing 

Pulse  stuffing  has  been  widely  applied  as  an  approach  for  time-division  multi- 
plexing of  asynchronous  bit  streams.  In  this  technique,  the  nominal  data  rate  of  the  output  of 
the  multiplexer  is  chosen  to  be  higher  than  the  sum  of  the  nominal  data  rates  of  the  inputs  to 
the  multiplexer.  The  data  on  each  of  the  input  data  channels  is  written  into  an  elastic  buffer 
and  withdrawn  as  required  by  the  multiplexer.  Occasionally,  a buffer  will  be  low,  and  thus  no 
data  will  be  read  out.  Instead,  a stuff  bit  is  transmitted.  These  stuff  bits  are  identified  by  sync 
signals  which  are  transmitted  with  the  data.  These  sync  signals  are  used  by  the  demultiplexer 
in  removal  of  the  stuff  bits.  The  insertion  and  removal  of  stuff  bits  results  in  timing  jitter  which 
must  be  reduced  by  using  a narrowband  phase-locked  loop  to  generate  the  READ  clock  for  the 
elastic  buffer. 

There  are  four  basic  steps  one  must  take  in  designing  a pulse  stuffed  multiplexer. 

1 . Select  the  multiplexer  data  rate  sufficiently  high  to  accommodate  the 
expected  frequency  uncertainty  of  the  data  channels  and  the  multiplexer 
clock. 

2.  Allocate  adequate  overhead  to  have  a very  reliable  indicator  of  framing 
and  stuff  bit  locations. 
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3.  Determine  buffer  sizes  so  that  buffer  overflow  will  not  occur  with  worst-case 
frequency  offsets  and  worst-case  waiting  times. 

4.  Make  the  loop  bandwidth  of  the  PLL  generating  the  read  clock  for  the 
destuff  buffer  small  enough  to  remove  timing  jitter  caused  by  the  stuff- 
destuff  process . 

The  disadvantages  of  the  technique  are,  first,  that  it  is  inconvenient  to  use  to 
synchronize  a large  number  of  lower  capacity  channels  for  time  division  switching.  This  is  due 
to  the  significant  amount  of  processing  per  channel  that  is  required  (such  as  buffering,  stuffing, 
destuffing  and  filtering  to  remove  timing  jitter).  Second,  a stuff -destuff  device  would  be 
required  at  every  digital  channel  terminal,  at  every  channel  termination  on  a digital  switch  and 
at  every  channel  input  to  a multiplexer.  Based  on  these  shortcomings,  pulse  stuffing  was  not 
considered  further  as  a network  synchronization  technique. 

Independent  Clocks 

Another  approach  for  using  unlocked  clocks  involves  using  a very  stable  independent 
clock  at  each  node.  All  subscribers  at  a node  are  then  slaved  to  this  nodal  clock.  Thus,  all 
data  arriving  at  a node  from  subscribers  will  be  synchronized.  Data  arriving  via  trunks  from 
other  nodes  having  independent  clocks  must  be  buffered.  This  buffer  must  be  sized  to  accommodate 
the  possible  frequency  offsets  between  the  independent  clocks  and  variations  in  frequency  due 
to  transmission  phenomena. 

If  the  nodal  clocks  are  set  to  run  at  a nominal  data  rate  of  f and  the  data  rate  of 

o 

any  two  nodal  clocks  is  within  Af  of  f , then  the  maximum  difference  in  data  rate  between  any 

' o o 

two  nodal  clocks  is  2Af  . Assuming  that  a buffer  size  of  m bits  is  chosen  (set  initially  at  half 

o 

full)  the  time  required  for  a buffer  overflow  is  at  least 


letting 


m/2 

2 A f 
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denote  the  long  term  clock  stability  factor  (1)  becomes 


Then  using  Equation  (3)  the  buffer  size  which  guarantees  that  the  time  between  overflows  or  under- 
flows is  at  least  T seconds  can  be  computed.  If  one  had  an  atomic  frequency  standard  with 
S = 10  ^ , then  using  Equation  (3)  we  find  that  a buffer  size  of  1040  bits  is  required 
(at  f =10  Mb/s)  to  guarantee  a period  of  30  days  between  buffer  overflows.  Thus,  one  can 
see  that  buffer  sizes  may  be  significant  with  this  approach  even  using  atomic  clocks. 

There  are  two  disadvantages  that  the  independent  clock  technique  has  vis  a vis 
any  of  the  disciplined  clock  approaches.  First,  because  clocks  are  not  disciplined,  ultimate 
inherent  accuracy  must  be  demanded  from  individual  clocks.  Thus,  the  number  of  cesium  clocks 
required  in  the  network  will  be  significantly  larger  than  any  of  the  locked  clock  approaches. 

This  results  in  a significant  cost  impact.  Second,  despite  the  highly  accurate  clocks,  periodic 
bit  slips  must  be  accepted  since  clocks  are  not  disciplined. 

Despite  these  basic  disadvantages,  the  independent  clocks  approach  was  not 
eliminated  as  a viable  candidate  partly  because  of  certain  other  inherent  advantages  and  partly 
because  the  effort  necessary  to  analyze  the  network  performance  of  this  basically  uncomplicated 
technique  was  small  as  compared  to  the  locked  clock  approaches  which  required  an  extensive 
simulation  effort. 


Synchronous  Techniques 

Three  techniques  bases  on  disciplined  clocks  have  been  explored.  In  this  section, 
a brief  overview  of  each  technique  is  presented  with  the  detailed  in-depth  analysis  deferred  to 
Section  3.0. 

The  inherent  characteristic  common  to  any  synchronous  clock  technique  is  that  the 
nodal  clock  at  each  node  within  the  network  is  disciplined  so  that  its  frequency,  phase  or  both 
is  maintained  within  some  small  tolerance  of  the  like  parameters  of  some  reference  timing  signal . 
The  fundamental  differences  between  the  three  techniques,  master-slave,  mutual  synchronization 
and  time  reference  distribution  is  the  choice  of  the  reference  timing  signal. 


Master-slave  is  a very  straightforward  approach.  Timing  is  distributed  through  a 
master-slave  tree.  The  nodal  clock  is  derived  by  locking  a very  narrowband  phase-locked  loop 
to  the  received  bit  timing  from  one  of  the  incoming  trunks.  This  approach  has  been  used  in  many 
of  the  commercial  systems  that  have  been  built  (Western  Union,  Datran,  and  AT&T).  Generally, 
a hierarchial  approach  with  a number  of  diverse  routes  for  timing  distribution  has  been  used. 

This  gives  an  added  measure  of  survivability,  since  it  minimizes  the  probability  that  a given 
node  will  be  forced  to  operate  asynchronously  due  to  a failure  in  the  particular  link  from  which 
it  derives  timing.  An  added  degree  of  sophistication  that  is  feasible  with  respect  to  alternate 
routing  is  adaptive  reorganization.  This  feature  is  compatible  with  either  master-slave  or  time 
reference  distribution  since  both  are  based  on  hierarchical  tree  structures.  However,  since  it 
requires  a duplex  control  link  and  such  a link  is  already  part  of  the  time  reference  distribution 
system  and  not  a part  of  master-slave  system,  this  feature  has  been  incorporated  in  the  time 
reference  distribution  approach. 

Mutual  synchronization  is  a synchronous  technique  where  the  reference  for  the 
local  nodal  clock  is  an  average  of  the  timing  signals  derived  from  each  of  the  incoming  trunks. 
Obviously,  the  loss  of  a single  i!nk  will  not  necessitate  a network  reorganization  os  it  might  in 
a master-slave  system.  However,  unlike  a master-slave  system,  a link  perturbation  on  any  of 
the  incoming  trunks  can  have  an  effect  on  the  nodal  clock.  In  general,  it  can  be  said  that  a 
common  average  frequency  for  all  nodes  in  the  network  is  achieved  in  the  mutual  synchronization 
system  at  the  expense  of  significant  variations  in  instantaneous  frequencies  between  nodes. 

Time  reference  distribution  is  more  closely  related  to  a master-slave  system  titan 
any  other  technique.  In  fact,  it  can  be  described  as  a double-ended  master-slave  system  that  is 
referenced  to  some  standard  of  precise  time  and  includes  both  an  adaptive  reorganization 
capability  and  a separation  of  error  measurement  and  error  correction.  This  point  is  worthy  of 
■’loboration . In  order  to  elaborate  on  this  statement,  four  points  must  be  discussed.  These  are: 

• Double-ended  versus  single-ended 

• Adaptive  reorganization 

• Reference  to  precise  time 

• Separation  of  error  measurement  and  error  correction 
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As  the  techniques  have  been  defined,  master-slave  is  a single-ended  system  and  TRD 
is  a double-ended  system.  The  single-ended  approach  uses  a reference  timing  signal  local  to  the 
clock  being  disciplined.  That  is,  the  reference  signal  is  the  timing  signal  derived  from  one  of 
the  received  trunks.  This  signal  is  essentially  the  timing  signal  generated  at  some  distant  node, 
corrupted  by  the  link  variations  inherent  in  the  media  and  corrected  by  the  appropriate  filtering 
in  the  receive  node.  Such  a system  does  not  require  an  overhead  link  for  timing  information. 

The  double-ended  system  attempts  to  lock  the  local  nodal  clock  directly  to  the  timing  signal 
generated  at  the  distant  disciplining  node.  This  is  accomplished  in  TRD  by  measuring  the  phase 
offset  between  clocks  via  a time  transfer  technique*  and  correcting  the  disciplined  clock  so  that 
the  measured  phase  difference  between  clocks  is  driven  to  zero.  The  time  transfer  technique 
requires  an  overhead  link  from  the  disciplining  node  to  the  disciplined  node.  The  reader  is 
directed  to  Section  3.0  for  a detailed  discussion  of  the  implementation  of  the  technique. 

The  adaptive  reorganization  capability  included  in  the  time  reference  distribution 
approach  addresses  the  problem  of  how  nodes  within  the  network  attain  a timing  reference  after 
the  primary  reference  has  gone  down.  The  adaptive  reorganization  approach  essentially  results 
in  a real-time  rating  of  all  the  potential  timing  reference  sources  available  for  a node.  This 
rating  is  based  on  link  qualities,  clock  types  ond  timing  distribution  paths  to  the  ultimate  master. 
The  technique  guarantees  that  the  best  available  reference  for  each  node  is  selected  in  case  of 
failure.  The  requirement  placed  on  the  network  for  this  service  is  an  overhead  channel  required 
for  the  distribution  of  link  and  nodal  rankings.  The  alternative  to  the  adaptive  reorganization 
scheme  is  a fixed  reorganization  scheme  that  can  result  in  deleterious  timing  results  in  the  face  of 
certain  combinations  of  failures. 

A valuable  by-product  of  the  TRD  approach  results  if  the  network  master  is  referenced 
to  a source  of  precise  time  such  as  UTC  at  the  Naval  Observatory.  If  this  is  done,  the  network 
will  now  have  available  at  each  node  a source  of  precise  time.  A double-ended  system  such  as 
TRD  is  the  only  type  that  can  provide  this  characteristic.  The  resulting  benefits  are  discussed 
elsewhere  in  this  report. 

The  overhead  link  required  in  the  TRD  approach  for  time  transfer  and  adaptive 
reorganization  can  also  be  used  for  the  transmission  of  measured  but  uncorrected  clock  errors  from 

* See  Paragraph  3.3 
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node  to  node.  As  is  shown  in  Section  3.0,  the  TRD  approach  provides  a natural  division  between 
error  measurement  and  error  correction.  If  the  time  interval  for  error  correction  is  long,  the 
measured,  but  uncorrected,  error  on  a series  of  tandem  links  may  be  provided  via  overhead  to 
nodes  lower  in  the  hierarchy.  This  permits  the  simultaneous  correction  of  all  clocks  in  the  network 
directly  to  the  master  clock. 

This  section  presented  brief  overviews  of  the  candidate  techniques.  Perhaps  the 
major  point  being  conveyed  in  this  section  dealt  with  the  similarities  and  differences  between  the 
three  synchronous  techniques.  In  summary,  any  synchronous  network  technique  must  deal  with 
two  specific  problems:  The  loss  of  the  reference  timing  signal  which  if  not  replaced  results  in 
asynchronous  operation  and  media  link  variations  which  result  in  a frequency  difference  between 
the  transmitted  timing  signal  at  the  disciplining  node  and  the  received  timing  signal  at  the 
disciplined  node.  The  first  problem  is  addressed  similarly  by  master-slave  and  TRD  (although  in 
different  levels  of  sophistication)  by  the  substitution  of  another  reference.  Mutual  synchronization 
addresses  the  problem  by  at  no  time  having  a single  nodal  input  constitute  the  reference  timing 
signal  for  a node.  In  case  of  link  variations,  master-slave  and  mutual  synchronization  have 
similar  approaches  and  TRD  takes  a completely  different  approach.  The  goal  with  respect  to  link 
variations  is  to  not  have  such  variations  affect  the  frequency  of  the  disciplined  nodal  clock.  If 
this  goal  is  achieved,  the  buffer  requirement  on  any  link  in  the  network  is  minimized  since  the 
accumulation  of  frequency  errors  on  a series  of  tandem  links  is  avoided.  The  approach  taken  by 
mutual  synchronization  and  master-slave  is  essentially  to  filter  out  all  but  very  long  term  variations. 
The  approach  taken  by  TRD  is  to  remove  all  such  media  effects  via  the  double-ended  time  transfer 
approach. 
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3.0  BASELINE  DESIGNS  FOR  DISCIPLINED  TECHNIQUES 

The  three  disciplined  techniques  being  considered  are  master-slave,  mutual 
synchronization,  and  time  reference  distribution.  In  this  section  we  perform  tradeoffs  leading 
to  the  selection  of  a good  approach  for  implementing  each  of  these  techniques.  We  do  not 
make  the  claim  that  the  design  parameters  are  optimum,  only  that  they  are  "good"  considering 
the  environment  in  which  the  timing  subsystem  must  operate. 

This  was  done  to  provide  a concrete  baseline  by  which  to  compare  the  disciplined 

techniques. 

3. 1 Master-Slave  Tradeoffs 

The  tradeoffs  for  a master-slave  system  involve  loop  type,  loop  parameters,  phase 
detector  type,  clock  stabilities,  buffer  sizes,  and  method  of  providing  alternate  references 
considering  the  expected  stress  environment  in  which  the  network  must  operate. 

3.1.1  Master-Slave  Block  Diagram 

A typical  block  diagram  of  the  nodal  timing  subsystem  for  the  master-slave 
technique  is  shown  in  Figure  3.1.1.  The  selector  on  the  left  selects  the  source  to  which  the 
nodal  clock  is  to  be  slaved.  The  inputs  to  this  selector  consist  of  clocks  generated  from  incoming 
data  links  or  from  external  reference  sources  such  as  USNO  or  Loran  C.  The  selector  on  the 
right  is  used  to  select  one  of  the  redundant  nodal  clocks  as  input  to  the  frequency  synthesizer. 
The  frequency  synthesizer  generates  all  the  frequencies  needed  for  switching  and  multiplexing 
equipment  at  the  node.  Tv/o  redundant  phase-lock  loops  are  shown,  however,  more  frequency 
sources  may  be  added  for  increased  redundancy.  The  actual  number  of  frequency  sources 

t 

is  determined  by  the  required  nodal  availability  and  reliability  of  the  basic  frequency  sources. 
Redundancy  of  other  nodal  timing  subsystem  hardware  is  not  shown  in  order  to  avoid  cluttering 

i 

( the  diagram.  All  of  the  frequency  sources  are  phase-locked  to  the  selected  input  frequency 

source.  This  is  done  so  that  in  the  event  of  a local  clock  failure  a smooth  transition  can  be 
made  from  one  local  source  to  the  next.  Each  of  the  VCO's  can  be  allowed  to  coast  through 
link  outages  through  a command  from  the  control  section  to  force  holding  the  VCO's  control 
voltage  at  the  same  value  as  immediately  before  the  dropout  for  the  duration  of  the  dropout. 
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Figure  3.1.1.  Master  Slave  Nodal  Timing  Subsystem  Block  Diagram  (Dual  Redundant) 


The  control  section  provides  operational  control  of  the  timing  subsystem,  reconfigures  the  redun- 
dant hardware  in  case  of  failures  and  controls  selection  of  different  links  or  references  to  phase 
lock  the  local  clock  to  in  case  of  changes  in  link  status.  Its  inputs  consist  of  nodal  fault  detec- 
tion signals,  local  command  signals,  link  dropout  signals,  and  remote  node  status  signals  via  order- 
wire  channels.  The  input  orderwire  information  can  be  used  to  implement  an  adaptive  timing 
distribution  network  reorganization  scheme  for  slaving  the  nodal  clock  to  the  best  available  source. 
This  type  of  adaptive  scheme  should  be  used  in  large-scale  military  networks  to  achieve  surviv- 
ability in  stress  situations.  A discussion  of  this  problem  is  given  in  Paragraph  8.1.  Typically,  a 
network  master  frequency  source  uses  a primary  standard.  Thus,  three  cesium  frequency  standards 
might  be  used  in  a triply  redundant  configuration  to  achieve  reliability.  To  achieve  greater  sur- 
vivability of  network  synchronization,  one  might  also  provide  several  slave  nodes  with  atomic 
frequency  standards  to  act  as  masters  if  the  network  master  is  lost.  It  is  probably  desirable  to 
slave  the  network  master  to  a common  reference  frequency  source  such  as  U.S.  Naval  Observatory 
(USNO).  This  will  allow  better  interoperability  with  other  networks  referenced  to  USNO. 

3.1.2  Design  Tradeoffs 

Consider  the  baseband  phase-locked  loop  model  shown  in  Figure  3.1.2.  The  closed- 
loop  transfer  function  of  the  phase-locked  loop  may  be  written  as 
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The  output  of  the  phase  detector  may  be  written  as 
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frequency  may  be  written  as 
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(3. 1.2-3) 


A type  one  loop  is  obtained  with  a simple  low-pass  loop  filter , i .e. , 
F(s)  = a/(s  + a).  In  this  case  the  closed-loop  transfer  function  becomes 
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A type  two  loop  is  obtained  by  using  an  integral  plus  proportional  loop  filter,  f.e.. 


p (s)  = (s  + a)/s . Then  the  closed  loop  transfer  function  becomes 

9o(s)  = 2 + 

s2  + 2 Zu  s + cj  2 
n n 


(3. 1.2-5) 


where  -Jo  Kv  and  (.  - V^'4  0 

Phase-locked  loops  with  these  two  loop  types  will  be  analyzed  in  this  section  for 
application  to  the  network  synchronization  problem.  They  have  considerably  different  properties 
in  some  respects,  but  one  car,  also  obtain  a blend  of  the  properties  of  both  by  using  the  loop 


filter  F(s)  = 


a(s  + b) 


3. 1.2.1  Type  One  Loop 

A linear  analysis  will  be  presented  (no  cycle  slips  occur  in  the  loopl . Since  it  turns 
out  that  this  is  a desirable  implementation,  the  results  are  valid.  Therefore,  it  is  assumed  that  a 
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phase  detector  that  is  linear  over  several  cycles  is  used  (an  extended-range  phase  detector!.  I he 
parameter  tradeoffs  that  will  be  examined  are  all  closely  interrelated. 
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The  purpose  of  the  nodal  synchronizer  is  to  provide  a stable  nodal  clock  that  closely 

follows  the  network  frequency.  Variations  in  frequency  due  to  path  delay  variations  should  be 

filtered  out.  It  is  assumed  that  a narrowband  loop  will  be  used  for  this  purpose.  Variations  with 

a period  of  a few  minutes  will  be  filtered  by  such  a loop.  The  most  troublesome  variations  are 

those  with  a daily  period  (cu.  - 73  x 10  6 rad/sec).  For  cable  and  microwave  the  equivalent 

d 10 

frequency  offsets  due  to  daily  path  delay  variations  would  be  a few  parts  in  10  . Satellite  links 

g 

present  much  worse  path  delay  variations  (a  few  parts  in  i0  ),  but  it  is  safe  to  assume  that  Doppler 
correction  can  be  used  to  provide  a reference  with  much  greater  stability  (even  if  this  correction 
was  in  error  by  1 percent,  the  resulting  stability  would  be  a few  parts  in  10  ^).  This  could  be  done 
by  using  the  satellite  orbital  parameters,  the  ground  terminal  locations,  and  the  time  to  calculate 
the  impact  of  the  satellite  motion  on  the  phase  error  and  to  use  this  result  to  correct  the  measured 
phase  error.  Since  the  network  master  would  be  no  more  accurate  than  ±10  , the  effects  of 

daily  path  delay  variations  need  to  be  attenuated  by  about  a factor  of  10  to  make  them  about  ihe 
same  magnitude  as  the  master  clock  variations.  We  should  point  out  that  it  is  more  important  to 
maintain  the  nodal  frequency  very  accurately  when  a reference  is  lost  than  to  heavily  filter  these 
slow  path  delay  variations  when  a reference  is  available.  The  problem  associated  with  trying  to 
filter  these  variations  at  all  times  is  the  long  acquisition  time  and  the  large  steady-state  phase 
error  that  must  be  built  up  to  hold  the  VCO  in  lock.  This  loop  requires  a nonzero  phase  error  to 
track  a frequency  offset.  From  Equation  (3. 1 .2-2)  this  phase  error  may  be  written  as 
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For  a step  in  frequency,  0.(s)  = A w/s2  . The  final  value  theorem  shows  that  the  steady-state 
phase  error  is 
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(3. 1.2-7) 


Thus,  the  peak  phase  error  that  must  be  accommodated  by  the  phase  detector  is  proportional  to 
£ and  inversely  proportional  to  the  natural  frequency  of  the  loop. 

Smaller  values  of  £ tend  to  speed  acquisition  time  (down  to  £ = 0.707)  and  to 
reduce  steady-state  phase  error.  However,  the  value  of  required  to  provide  a given  amount 
of  filtering  of  the  input  frequency  jitter  decreases  for  smaller  values  of  £ which  tends  to  increase 
steady-state  phase  error.  Thus,  one  should  also  examine  this  effect. 

Using  a low-pass  loop  filter 


w 2 s 9 (s)  (3. 1.2-9) 

F (s)  - — x 

S +2?us  + « 2 
n n 

Then 


F (s) 
o 

F^y 


CO 

n 

“2 T 

s +2f  w s+O) 
n n 


This  transfer  function  has  magnitude 


(3.1.2-10) 
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fo  filter  daily  jitter  by  a factor  of  10  it  is  required  that 


F„« 

CJd) 

Fi<i 

= 0.1. 


The  value  of  a>  reauired  fo  achieve  this  is  a function  of  One  can  see  from  Figure  3.1  .2.1 
n 

and  from  Equation  (3.1  .2-7)  that  as  long  as  the  ratio  increases  linearly  with  £ as  it  does  for 
£ >2.5  there  will  be  no  change  in  steady-state  phase  error.  But  for  £ <2.5  the  ratio 
increases  much  more  slowly  with  £ which  indicates  that  sma II  values  of  £ will  result  in  small 
steady-state  phase  error. 
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3. 1.2.1 


Value  of  w /w  Required  to  Make  H(a>)  0 . 1 as  a Function  of  £ 


The  value  or  £ = 0.707  gives  fhe  shortest  acquisition  time  for  constant  a>  . This  is  nor 
exactly  rhe  case  here  since  rhe  value  of  at  required  ro  maintain 
increases  as  £ increases.  However,  the  increase  is  not  rapid  enough  to  make  higher  values  of 
£ give  better  acquisition  time.  In  addition,  values  of  £ less  0.707  greatly  increase  the 
oscillatory  nature  of  fhe  loop  response,  and  these  variations  would  be  passed  on  to  other  nodes. 
Finally,  the  value  £ - 0.707  gives  the  smallest  steady-state  phase  error  for  any  value  of 
£ ^ 0.707.  Thus,  the  value  of  £ = 0.707  appears  to  be  the  best  choice  for  this  loop. 

Assuming  that  the  effects  of  daily  path  delay  variations  will  be  heavily  filtered  (requiring 

a)  = 0.32  co.)  and  that  s 0.707  will  be  used,  the  requirements  on  phase  defector  range  can 
n d 

be  examined.  There  is  a direct  trade  between  oscillator  stability  and  phase  d.  ector  range.  The 
maximum  phase  error  is  about  1.05  € due  fo  the  slight  overshoot  with  ( - 0.707.  The  steady- 
state  phase  error  is  given  by  Equation  (3.1  .2-7).  From  this  equation  the  long  term  clock  stability 
required  fo  keep  fhe  maximum  phase  error  less  than  any  amount  can  be  calculated  (at  a specific 
reference  frequency)  from 

Ru) 

S = -A".  = /U  , (3.1.2-11) 

co  o . I f;  co 

where  R is  fhe  phase  detector  range  ( ± R radians!.  At  reference  frequencies  of  several  megabits 

and  ± tt  radian  range,  this  will  result  in  a stability  requirement  of  a few  parts  in  10  (which 

-8 

cannot  be  achieved  by  quartz  clocks).  One  could  ease  this  requirement  (to  10  ) by  extending 

the  range  of  fhe  phase  detector  fhe  required  amount.  This  approach  would  also  require  buffers  of 

similar  size,  and  these  would  add  several  milliseconds  of  additional  delay  at  every  node  that  would 

otherwise  not  be  requited.  The  ether  alternative  is  fo  increase  w and  use  a much  smaller  phase 

n 

detector  range.  In  this  case  the  r do!  frequency  would  be  significantly  affected  by  path  delay 
variations  and  this  would  degro  -.e  performance  when  the  loop  must  "coast"  without  a reference 
since  it  would  "coast"  at  this  perturbed  frequency.  In  contrast,  fhe  type  two  loop  filter  has  an 
integrator  which  car.  be  designed  fo  have  a very  long  time  constant.  The  integrator  output  provides 
the  VCO  with  a control  voltage  which  is  a much  more  accurate  measure  of  fhe  true  network 
frequency . 


F (jco,)/F.(j<u  ) =0.1 

o d i d 
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3. 1.2. 2 


Type  Two  Loop 

Two  principal  differences  of  this  type  of  loop  are  that  the  integrators  in  the  loop 
filter  provides  "frequency  memory"  for  use  when  "coasting"  without  a reference  and  that  the  loop 
bandwidth  can  be  changed  as  an  acquisition  aid  to  speed  up  acquisition  from  large  frequency  off- 
sets. The  loop  parameters  used  in  tracking  do  not  impose  a limit  on  the  acquisition  time  as  they 
do  when  using  a type  one  loop.  This  will  allow  the  use  of  very  narrowband  loops  while  still 
retaining  good  acquisition  performance . Several  approaches  have  been  examined  in  Appendix  C, 
and  some  of  the  results  are  used  in  this  section.  We  will  assume  the  two-bandwidth  approach  is 
used.  It  should  be  rather  easy  to  implement  in  a digital  loop. 

An  extended  range  phase  detector  improves  the  performance  of  the  loop  significantly 
in  this  application  by  allowing  the  use  of  narrower  bandwidfhs,  and  it  will  be  assumed  that  it  is 
used.  The  phase  detector  must  have  a range  larger  than  the  peak  phase  error  that  will  ever  be 
seen.  This  will  ensure  that  the  loop  operates  in  the  linear  region.  The  manner  in  which  the 
performance  is  improved  over  the  phase  detector  with  a one-cycle  range  is  in  improved  acquisition 
time  and  ability  to  acquire  greater  frequency  offsets  for  identical  £ and  at  . For  any  frequency 
offset  outside  the  "lock  range"  of  the  one-cycle  phase  detector,  this  loop  will  slip  cycles  and 
require  longer  to  acquire.  The  acquisition  time  of  the  one-cycle  phase  detector  becomes  poorer 
and  poorer  relative  to  the  extended  range  phase  defector  as  the  offset  is  increased  until  the  offset 
becomes  larger  than  the  loop  capture  range.  At  this  point,  the  one-cycle  phase  detector  cannot 
even  achieve  lock.  To  make  this  loop  have  acquisition  times  similar  to  those  of  the  extended 
range  phase  defector  would  require  a considerably  larger  loop  bandwidth,  and  attaining  a very 
narrow  loop  bandwidth  Is  one  of  the  principal  goals  for  the  network  synchronization  problem. 

There  is  one  other  useful  feature  of  the  extended  range  phase  detector.  Since  the  fype  two  loop 
tracks  a frequency  offset  wirr.  zero  sready-state  phase  error,  and  if  does  not  slip  cycles  with  an 
extended  range  phase  defector,  then  each  nodal  clock  will  stay  close  in  phase  to  the  clock  to 
which  if  is  referenced.  This  helps  keep  the  probability  of  buffer  overflows  small. 

The  purpose  of  the  using  the  narrowband  loop  is  to  heavily  filter  the  jitter  caused  by 
perfubations  in  the  network  such  as  path  delay  variations  so  that  each  nodal  clock  is  very  stable. 
The  type  two  loop  has  a peaked  closed  loop  transfer  function  so  there  will  be  amplification  of 
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some  jitter  frequencies.  The  amount  of  amplification  is  easy  to  estimate.  From  Equation  (3.1  .2-5) 
we  can  determine  the  magnitude  of  the  frequency  transfer  function 


F (j  oj) 

2 2 /4  f 2 

OJ  + oj  s 

o ’ 

= 2 Uj 

n 

n 

F.(j  oj) 

+ M ^ 2 2)  2 2 4 

L 14  s *>ojn  OJ  +OJn  J 

1/2 


(3.1.2-12) 


The  maximum  value  of  this  transfer  function  has  been  determined  and  is  shown  in  Figure  3.1  .2.2. 
There  is  considerable  amplification  of  some  jitter  frequencies  at  the  smaller  values  of  £ • For 
example,  at  £ = 0.707  the  peak  occurs  at  oj  C.78( , and  that  jitter  is  amplified  by  a 
factor  of  1 .27.  For  ten  such  nodes  in  tandem,  this  jitter  would  be  amplified  by  a factor  of 
(1 .27)^  = 10.9.  This  is  clearly  unacceptable,  but  the  problem  can  be  alleviated  by 

choosing  larger  values  for  damping  coefficient  £ . For  example,  at  £ = 4 the  peak  occurs  at 

oj  = 0.4W  , and  that  jitter  is  amplified  by  a factor  of  only  1 .013 . For  ten  of  these  nodes  in 

n 10 

tandem,  the  jitter  would  only  be  amplified  by  a factor  of  (1 .013)  =1.14.  Thus,  with  this  looo 

the  higher  values  of  £ will  have  to  be  used. 

In  selecting  parameters  for  the  narrowband  loop,  one  can  examine  two  different 

approaches.  First,  one  could  artempt  to  filter  the  effect  of  daily  variations  all  the  time.  This 

would  require  extremely  narrowband  loops,  and  the  loops  would  be  extremely  slow  to  respond  to 

any  variations  in  reference  frequency.  On  the  other  hand,  one  could  require  only  that  the  loop 

heavi  ly  filters  the  effects  of  daily  variations  on  its  frequency  memory  and  thus  on  its  frequency 

when  in  the  "coasting"  mode  (operating  without  a reference).  By  this,  we  mean  that  effects  of 

path  delay  variations  on  the  integrator  output  are  heavily  filtered.  To  do  this  one  must  ensure 

that  the  time  constant  integrator  is  long  enough  for  the  integrator  output  to  closely  reflect  the 

network  frequency  averaged  over  the  past  several  days.  This  will  allow  a much  larger  a>  to  be 

n 

used,  and  this  approach  accomplishes  the  critical  task  of  providing  a nodal  clock  with  frequency 
extremely  close  to  the  network  frequency  during  loss  of  reference. 

If  one  desires  to  heavily  filter  the  jitter  at  all  times,  the  effect  of  the  filter  is  given 
by  the  frequency  transfer  function  Equation  (3.1.2-12).  Assuming  that  we  wish  to  make 

I H(w)  | 0.1  the  value  of  oj  / oj  required  to  achieve  this  as  a function  of  £ is  shown  in 

I n 

Figure  3. 1 .2 . 1 . Note  the  rapidity  with  which  the  value  of  o>  required  to  achieve  the  filtering 
decreases  with  increasing  £.  A large  value  of  £ must  be  used  or  other  jitter  frequencies  will  be 
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greatly  amplified.  Even  at  a value  of  £ = 2 .5  the  va lue  of  required  to  attenuate  dai ly 

iitter  by  a factor  of  10  is  (u  = 0.02  Co  . 

1 ' n d 

In  contrast,  assume  that  one  only  attempts  to  filter  the  effects  of  daily  jitter  on  the 
integrator  output.  The  output  frequency  is  determined  only  by  the  voltage  on  the  integrator  which 
is  the  integrated  past  history  of  the  phase  error,  E(s).  Then  the  output  frequency  is 

aK 

F (s)  - ± E(s) 

o s 


Cl) 


F.  (s) 

i 


(3.1.2-13) 
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s +2£a>s  + oj 

n n 


The  magnitude  of  the  frequency  transfer  function  is  identical  to  Equation  (3.1 .2-10).  The  value  ofcJ /u 
required  to  make  H(oj)|  ~ G.  1 has  been  shown  previously  in  Figure  3.  1 . 1 as  a function  of  £. 

We  note  that  in  contrast  to  the  previous  example  that  as  one  increases  £ that  tu  can  be  increased. 


In  fact,  for  large  £ one  can  niake 


H(tu.) 


= K by  choosing  a>  « 2 f K w..  If  we  assumed  that 
n ' I 

operation  with  values  of  f in  the  range  from  3 to  5 one  can  increase  oj  by  a factor  of  36  to  100 
by  only  requiring  that  the  effect  of  daily  jitter  on  the  integrator  output  be  filtered. 

Thus,  we  are  recommending  the  use  of  a type  two  loop  with  u> ^ and  £ chosen  to 

heavily  filter  the  effects  of  daily  jitter  only  on  the  integrator  output.  As  an  example,  if  we  pick 

£ - 4,  (1J  = 0.8  w is  needed  to  attenuate  the  daily  jitter  by  a factor  of  10.  From  Figure 

n d 

3.1  .2.2  we  find  that  the  peak  amplification  of  any  jitter  frequency  is  1 .013  and  this  occurs  at 

frequency  a>  0.32  <u  (which  corresponds  to  buffer  fill  for  the  extended  range  phase  detector) 

d 

from  Figure  3.1  .2.2.  Since  this  loop  is  overdamped  (6  > 1),  the  phase  error  (in  cycles)  caused 
by  a frequency  offset  is 
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For  large  £ , 
is  found  to  be 


2 4 w and  ar_  ss  ro  / 2 4 . The  time  at  which  peak  phase  error  occurs 


H n a ^ - P n «2 


(3.1 .2-15) 


and  the  value  of  0 (t  ) is 
e m 


9 (t  ) 

e m 


a2  ^ ^ +a2  ^ (T-a^-a^  / a a +a^/cr  , (3.1.2-16) 


where  cr  = - a^.  For  these  loop  parameters  the  phase  error  gradually  rises  to  a peak  with 

time  constant  T,  = 1/  . This  peak  occurs  after  a little  over  four  time  constants  or  about  2.5 

c*l 

hours.  The  loop  then  slowly  reduces  this  phase  error  to  zero  with  a time  constant  of  t = 1/ 

1 . 2 

or  about  37.5  hours.  The  second  time  constant  is  the  time  constant  associated  with  the  integrator 

output.  The  peak  phase  error  caused  by  a step  in  frequency  decreases  with  increasing  4 (for 

constant  to  ).  The  reason  for  this  is  that  the  two  time  constants  get  further  apart  with  t-  decreasing 
n I 

and  increasing.  This  decrease  in  allows  the  loop  to  more  rapidly  respond  to  changes  in 

reference  frequency,  thereby  preventing  the  accumulation  of  large  phase  errors.  For  these  loop 

_9 

parameters  perturbations  caused  by  fractional  frequency  steps  of  10  or  less  and  the  typical  path 
delay  variations  mentioned  previously  will  cause  maximum  phase  errors  of  no  more  than  about 

g 

1 ps.  Acquisition  from  large  frequency  offsets  of  several  parts  in  10  will  cause  much  larger 
peak  phase  errors  and  will  be  handled  by  switching  to  a special  acquisition  mode  with  a much 
wider  bandwidth  loop. 

The  tradeoffs  leading  to  the  choice  of  parameters  for  the  acquisition  mode  are  given 
in  Appendix  C,  but  a summary  of  the  parameters  is  given  here.  Switchover  to  the  acquisition  mode 
will  occur  when  the  phase  error  gets  larger  than  about  2 ps.  The  acquisition  loop  will  then  have 
to  respond  to  a frequency  step  and  a 2 ps  phase  step.  The  loop  bandwidth  should  be  chosen 


3-14 


**r 


wide  enough  to  allow  a rapid  pull-in,  but  it  should  not  be  too  wide  or  the  frequency  transient 
due  to  the  2 ps  phase  step  will  be  very  large.  The  peak  frequency  change  due  to  a phase  step 
for  the  type  two  loop  is 


f = 2 $ a>  A . (3.1.2-17) 

\p  n 

The  acquisition  mode  parameters  selected  were  * 0.707  and  0 .007  rad/sec  . With 

these  parameters  and  a 2 ps  initial  phase  offset  then  the  ooserved  \f  corresponds  to  a 

-9  ^ . 

fractional  instability  of  6.4  x 10  . This  is  about  the  same  order  of  magnitude  but  in  the  opposite 

direction  as  the  initial  frequency  offsets  that  could  cause  the  loop  to  go  into  the  acquisition  mode. 
Of  course,  if  one  desired  to  have  a smaller  frequency  change  due  to  the  phase  step,  it  could  be 
obtained  by  using  a narrower  loop  bandwidth  for  the  acquisition  mode.  One  must  remain  in  the 
acquisition  mode  long  enough  to  ensure  that  the  voltage  built  up  on  the  integrator  in  the  loop 
filter  will  hold  the  VCO  sufficiently  close  to  the  reference  frequency  to  avoid  another  buildup 
of  a 2 ps  phase  error  after  switchover  to  the  tracking  parameters.  To  be  conservative  we  have 
estimated  that  remaining  in  the  acquisition  mode  for  a total  of  9 / £ w seconds  is  more  than 
adequate.  For  these  parameters  this  corresponds  to  about  30  minutes  in  the  acquisition  mode. 

One  other  question  that  needs  to  be  examined  is  the  behavior  of  the  loop  when  the 
VCO  is  subject  to  a linear  frequency  drift.  From  Equation  (3.1  .2-2)  the  phase  error  of  the  type 
two  loop  can  be  written  as 


E(s)  = 


2 9 M 

s . (s) 

I 


(3.1.2-18) 
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Assume  that  the  frequency  drift  can  be  described  by 


(3.1.2-19) 


which  gives 


The  type  2 loop  responds  to  this  input  with  a constant  steady-state  phase  error.  This  error  is 
obtained  from  the  final  value  theorem  as 


= lim  sE(s) 
s — *-0 

■-A-  • 


(3.1.2-21) 


If  one  uses  a very  good  crystal  VCO  as  is  used  in  the  Bell  System  with  a drift  of  10  parts/day 
(or  1.16  x 10  ^ parts/second)  with  the  tracking  parameters  we  have  selected,  the  steady-state 

phase  error  is  found  to  be  0.37  ps.  This  is  small  enough  so  that  it  does  not  impact  our  choice 
of  the  2 ps  threshold  for  the  switchover  from  the  tracking  to  the  acquisition  mode. 

The  type  two  loop  is  much  more  suitable  for  this  application  than  the  type  one  loop. 
Its  most  useful  feature  is  "frequency  memory."  The  integrator  time  constant  can  be  selected  long 
enough  to  provide  heavy  filtering  of  daily  jitter.  Thus,  the  frequency  memory  provides  a very 
accurate  indication  of  average  network  frequency  over  a period  of  several  days.  The  type  two 
loop  can  also  be  used  with  a wider  acquisition  bandwidth  which  will  allow  much  faster  acquisition 
than  the  type  one  loop  in  the  presence  of  large  frequency  offsets.  This  will  allow  much  better 
performance  in  a military  environment  where  there  may  be  frequency  losses  of  the  reference  and 
significant  perturbations  in  network  frequency  under  stress  conditions. 


3. 1.2. 2. 


A summary  of  loop  parameters  for  the  master-slave  approach  is  given  in  Table 
Table  3.1  .2.2.  Loop  Parameters  for  the  Master-Slave  Approach. 
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It  was  mentioned  previously  that  a loop  filter  of  the  type  F (s) 


a(s  + b) 


b(s  -*  o) 

will  give  a loop  many  of  the  properties  of  both  types  of  filters  mentioned  previously.  By  proper 

choice  of  parameters  the  steady-state  error  can  be  made  considerably  smaller  than  that  obtained 

with  the  filter  F (s)  = — — . In  addition,  one  can  also  use  a two  bandwidth  filter  approach 

s + a 

to  provide  both  good  acquisition  and  good  filtering  performance.  However,  in  order  to  obtain  the 
type  of  behavior  which  we  consider  desirable,  one  will  have  to  select  parameters  so  that  the  per- 
formance very  nearly  approximates  that  of  the  type  two  loop  and  this  will  be  nearly  identical  to 
the  behavior  just  discussed.  The  parameters  for  this  loop  can  be  chosen  such  that  the  closed  loop 


transfer  function, 


HM 


< 1 , (j  >0.  Thus,  no  jitter  frequencies  will  be  amplified.  This  is 


significant  for  mutual  synchronization  systems,  so  this  type  of  filter  will  be  discussed  in  that  section. 


3.1.3  Survivability 

Survivability  indicates  the  degree  to  which  the  timing  subsystem  continues  to  perform 
its  function  (to  provide  network  timing)  during  periods  of  stress.  To  achieve  this,  fail-back  modes 
of  operation  must  be  provided  so  that  the  timing  subsystem  itself  will  not  cause  significant  degra- 
dation in  capability  of  the  network  to  provide  dependable  digital  communications  during  these 
periods.  Network  synchronization  is  necessary  because  the  slip  rate  that  two  nodes  will  experience 
in  communicating  is  directly  proportional  to  the  frequency  offset  between  th • i r nodal  clocks.  If 
this  offset  increases  in  a stressed  environment,  then  slip  rate  increases  which  leads  to  misaligned 
frames  in  multiplexers  and  switches  and  a need  to  resynchronize  cryptos.  Since  timing  subsystem 
performance  is  directly  tied  to  slip  rate,  slip  rate  as  a function  of  the  level  of  stress  indicates  the 
survivability  of  the  timing  approach. 

There  are  two  parts  to  the  problem  of  minimizing  the  slip  rate  in  a digital  network: 

1 . Keep  the  nodal  clocks  of  all  communicating  node  pairs  synchronous  as  much  as 
possible. 

2.  Minimize  the  frequency  offset  between  the  nodal  clocks  of  asynchronous 

communicating  nodes  and  minimize  the  amount  of  time  they  must  communicate 
with  asynchronous  nodal  clocks. 
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To  put  these  comments  in  perspective,  note  that  if  one  uses  the  independent  clock 
technique,  there  are  asynchronous  clocks  at  every  node.  The  slip  rate  due  to  buffer  overflows  is 
strictly  a function  of  the  clock  accuracy  and  the  buffer  size  and  will  not  change  as  a function  of 
stress  level.  On  the  other  hand,  the  analysis  in  Paragraph  8.1  indicates  that  one  could  reduce 
the  slip  rate  by  slaving  these  clocks  to  a common  master  through  some  master -slave  timing  distri- 
bution hierarchy.  This  would  make  all  nodal  clocks  synchronous  and  slips  due  to  buffer  overflows 
could  be  avoided  entirely  providing  there  were  no  link  or  node  failures.  If  link  or  node  failures 
occur,  some  nodes  may  lose  their  reference  to  the  master.  Then  some  nodes  with  asynchronous 
nodal  clocks  may  be  communicating  and  could  experience  slips.  Such  nodes  will  be  called 
"asynchronous  communicating  nodes."  These  nodes  will  remain  asynchronous  until  they  once  again 
become  locked  to  a common  reference. 

The  results  of  the  analysis  in  Paragraph  8.1  indicate  that  some  performance  improve- 
ments over  an  indpendent  clock  approach  may  be  obtained  by  using  one  or  more  fixed  references 
in  a master-slave  manner,  but  these  improvements  are  not  that  great  when  the  link  outage  probabil- 
ity is  high  because  the  timing  distribution  network  is  disrupted  so  frequently.  This  is  the  reason 
that  master-slave  systems  have  been  frequently  criticized  as  not  having  survivability.  However, 
one  can  make  an  additional  improvement  in  a master-slave  system  by  giving  it  the  capability  of 

adaptively  reconfiguring  the  timing  distribution  network  as  link  outages  occur.  This  technique 

92 

has  been  implemented  in  the  Dataroute  network  and  has  been  proposed  for  use  with  Time 

, 169,  170  . | I I ..  .. 

Reference  Distribution.  This  approach  has  the  properties  that: 

1 . All  communicating  nodes  are  synchronous  in  the  steady  state  (after  a sufficiently 
long  period  following  the  last  failure). 

2 . The  only  time  a nodal  clock  will  act  as  a self-reference  is  while  the  timing 
• distribution  network  is  being  reconfigured. 

The  use  of  an  adaptive  timing  distribution  network  gives  the  master-slave  approach  a very  high 
i degree  of  survivability.  The  system  can  be  designed  so  that  no  slips  occur  in  the  steady  state. 

The  only  time  slips  can  occur  is  during  the  transients  associated  with  reconfiguring  the  network. 
Obviously,  one  of  these  transients  occurs  every  time  an  outage  occurs,  and  any  single  outage  can 
cause  the  network  to  be  reconfigured  in  such  a manner  that  several  nodes  may  have  to  act  as  a 
self-reference  during  this  process.  Performance  in  a stress  environment  is  influenced  by  both  the 

3-18 


time  required  to  reconfigure  the  timing  distribution  network  and  the  length  of  time  a node  can 
act  as  a self-reference  without  experiencing  slips  (which  is  a function  of  the  buffer  size  and 
frequency  offset).  There  is  a trade-off  here  between  time  required  for  reconfiguration,  buffer 
size,  and  the  fractional  frequency  offset  between  the  nodal  clock  and  the  network  frequency. 

For  example,  a cesium  clock  could  operate  without  a reference  for  24  hours  while  accumulating 
only  about  I ps  of  phase  error  relative  to  the  network  master.  In  actual  practice,  though  it 
should  be  no  problem  to  accomplish  network  reconfiguration  in  10  minutes.  In  this  case,  even 
the  phase  errors  accumulated  with  quartz  clocks  will  be  rather  small.  The  analysis  of  Paragraph 

8.1  indicates  the  impact  of  these  parameters  and  for  practical  values  of  them  that  this  technique 
will  have  an  extremely  high  degree  of  survivability. 

3.2  Mutual  Synchronization  Trade-Offs 

In  this  section  we  address  the  implementation  of  a mutual  synchronization  approach 
for  network  synchronization.  Many  of  the  trade-offs  are  similar  to  those  discussed  in  the  previous 
section  with  respect  to  the  master-slave  approach.  These  include  the  decisions  one  must  make 
regarding  loop  type,  loop  parameters,  phase  detector  type,  clock  stabilities,  and  buffer  sizes. 
Additional  considerations  encountered  only  with  mutual  synchronization  include  the  individual 
phase  detector  weightings  and  the  system  stability. 

3.2.1  Mutual  Synchronization  Block  Diagram 

A block  diagram  of  the  nodal  timing  subsystem  for  the  mutual  synchronization 
technique  is  shown  in  Figure  3.2.1 . A dual  redundant  clock  system  is  shown  but  more  than  two 
clocks  can  be  utilized  to  increase  reliability  if  required.  The  blocks  labeled  "Phase  Detectors" 
contain  a phase  detector  for  each  input  frequency  source.  The  sources  are  obtained  from  incoming 
data  links  or  from  external  references  such  as  USNO  or  Loran  C.  The  phases  between  each  of  the 
incoming  frequency  sources  and  each  local  clock  are  then  weighted  according  to  some  relative 
quality  measure  of  path  and  ultimate  source.  Those  weighted  phase  differences  associated  with  a 
particular  VCO  are  then  summed  in  order  to  obtain  a composite  phase  difference.  This  composite 
phase  difference  is  then  smoothed,  filtered  and  amplified  by  the  loop  filter  circuit  having  frequency 
domain  transfer  function  F (s)  is  selected  to  filter  out  path  delay  variations  of  the  incoming  links 
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and  thereby  provide  a stable  local  source  that  is  phase  locked  to  the  long  term  average  of  the 
network  frequency.  The  outputs  of  the  voltage  controlled  oscillator  are  fed  back  to  the  phase 
detectors  and  also  applied  to  the  input  of  a selector  switch  which  serves  to  select  one  VCO  as 
input  to  a frequency  synthesizer  (which  generates  all  the  frequencies  needed  to  operate  the  nodal 
switching  and  multiplexing  equipment).  The  control  section  provides  a means  for  controlling  the 
operation  of  the  nodal  timing  subsystem,  for  reconfiguring  in  case  of  nodal  hardware  failures  and 
for  controlling  the  weightings  of  the  various  link  clock/local  clock  phase  differences.  The  inputs 
to  the  control  section  include  local  command  signals,  nodal  fault  defector  signal,  and  link  drop- 
out indicators  from  link  receivers.  The  mutual  sync  timing  subsystem  is  quite  similar  to  that  of 
the  master-slave  technique.  In  a manner  similar  to  the  master-slave  technique,  the  redundant 
local  clocks  are  all  phase  locked  to  the  network  frequency  so  that  smooth  transitions  can  be  made 
from  one  local  clock  to  the  next.  In  case  of  long  term  loss  of  links  the  control  section  will  deter- 
mine a new  set  of  phase  error  weightings. 


3.2.2  System  Stability 

Constraints  on  system  parameters  required  to  produce  a stable  mutual  synchronization 
between  have  taken  two  forms.  First,  one  must  ensure  that  the  relationship  between  path  delays 
and  loop  gain  is  proper.  This  results  in  upper  bounds  on  gain-delay  products  (loop  gains  and 
link  delays)  that  have  been  derived  by  many  investigators.  '°7'  40,  115,  45,  46  These 
stability  bounds  are  dependent  on  the  specific  network  configuration.  However,  the  maximum 
value  of  gain-delay  product  that  can  typically  be  tolerated  is  on  the  order  of  1 . Since  very 
narrowband  loops  (implying  very  small  loop  gain)  will  be  used,  link  delays  of  many  seconds  would 
be  required  before  there  would  be  a stability  problem.  Thus,  in  a real  network  one  would  not  have 
ro  be  concerned  about  the  gain-delay  products  being  large  enough  to  cause  network  instability. 

The  other  constraint  needed  to  ensure  stability  is  on  the  loop  filter  parameters. 
Stability  analyses  for  single-ended  mutual  synchronization  systems  have  shown  that  in  order  for  a 
mutual  synchronization  system  to  be  stable  requires  a closed  loop  phase  response  that  satisfies 


©0(i  cj  ) 
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I H(w  )[  < 1 , u)  > 0 


(3. 2. 2-1) 


3-21 


r 


1 


fi. 

* 


It  can  be  shown  that  type  two  loops  with  filters  of  the  form  F (s)  (s  + a)  / s never  satisfy  this 
condition  (we  note  that  I H(cu  ) >1).  This  means  that  phase  jitter  at  the  natural  frequency  w 

will  be  amplified  each  time  it  passes  through  the  timing  regeneration  circuitry  at  a node.  Since 
this  is  a closed-loop  system  of  many  interconnected  PLL's,  one  can  see  that  the  amplification  of 
phase  jitter  is  the  source  of  the  instability.  Thus,  the  designer  is  restricted  to  type  one  loops. 

One  could  examine  several  type  one  loops  for  use  in  this  system.  If  no  loop  filter 
is  used,  a first-order  loop  results.  It  is  obvious  from  the  transfer  function  that  such  a loop  satisfies 
condition  Equation  (3. 2. 2-1).  If  the  loop  has  a single  low-pass  filter  of  the  form  F (s)  = a/  (s  - a), 
then  one  can  show  that  the  closed-loop  transfer  function  Equation  (3.1 .2-4)  satisfies  condition 
Equation  (3. 2. 2-1)  providing  that  the  damping  coefficient,  £,  is  greater  than  1/ \J  2.  This  is 
the  loop  analyzed  in  Paragraph  3.1.2  for  the  master-slave  system,  and  many  of  the  same  conclusions 
apply  here,  but  to  use  this  loop  we  must  simply  ensure  that  the  damping  coefficient  is  always  large 
enough  to  give  a stable  system. 

Another  loop  filter  that  one  could  use  is  of  the  form  F (s)  - a(s  + b)  / b(s  +a).  This 
filter  gives  a type  one  loop,  but  its  performance  can  be  made  to  approximate  that  of  a type  two 
loop  by  making  the  pole  be  very  close  to  the  origin  relative  to  the  zero.  However,  the  require- 
ments for  stability  in  a mutual  sync  system  are  more  severe  than  those  for  the  simple  low-pass 
filter.  The  closed-loop  transfer  function  can  be  found  from  Equation  (3.1 .2-1)  as 
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function  is  then  found  to  be 
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To  find  the  conditions  under  which  a mutual  sync  system  with  this  filter  is  stable  we  must  deternvne 
the  conditions  under  which  H(  u)  < 1 for  w > 0.  Since  both  numerator  and  denominator 
of  Equation  (3. 2. 2-3)  have  an  4 term,  the  condition  Equation  (3. 2. 2-1)  implies  that  the 
requirement  for  stability  reduces  to 


2 2 2 4 

(2  £ - c)  u < w 


(4$  2 - 2)  u 2 u2;u  > 0. 
n 


(3. 2. 2-4) 


This  in  turn  reduces  to  the  requirement  that 


£ > (c2  + 2)  / 4c 


(3. 2. 2-5) 


Thus,  the  stability  requirement  leads  to  a lower  bound  on  the  acceptable  values  of  the  damping 
coefficient,  £ . Another  consideration  is  that  one  must  place  the  pole  of  the  loop  filter  closer 

to  the  origin  than  the  zero,  i.e.,  this  requires  b > a.  This  condition  leads  to  an  upper  bound 
on  the  acceptable  values  of  s 
One  can  write  the  parameter  b as 


b = 


2 * 


(3. 2. 2-6) 


b > a requires  that 


4 < (c2  + 1)  /2c 


(3.2.2 -7) 


The  desired  range  of  operation  of  this  loop  is  for  very  small  values  of  c.  Unfortunately,  one  can 
see  from  the  upper  and  lower  bounds  on  the  allowable  values  for  4 that  asymptotically  at  very 
small  values  of  c both  the  upper  and  lower  bounds  on  £ approach  1/2  c.  In  fact,  even  for 
c =0.1  the  allowable  range  for  £ is  found  to  be  5.03  < 6 < 5.05.  Of  course,  this  range  is 

much  too  small  to  allow  practical  implementation  of  the  loop. 


Thus,  because  of  stability  considerations  we  will  consider  only  loop  filters  of  the 
form  F(s)  = a/(s  + a)  for  use  in  mutual  sync  systems.  Design  considerations  for  this  type  of  loop 
are  discussed  in  the  following  subsection. 
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Design  considerations  for  a loop  with  a filter  of  the  form  F(s)  - a/(s  + a)  were 
treated  in  considerable  detail  for  use  in  a master-slave  system  in  Paragraph  3. 1 .2. 1 . Many  of 
the  same  results  apply  to  this  problem. 

In  that  analysis  we  found  that  one  cannot  use  a fast  acquisition  strategy  such  as  a 
two-bandwidth  filter,  but  one  can  make  good  use  of  an  extended  range  phase  detector  to  aid  the 
acquisition  process  and  provide  more  tracking  range.  We  also  found  that  if  one  specified  the 
amount  of  attenuation  at  a fixed  jitter  frequency  then  the  steady-state  phase  error  and  acquisition 
time  are  best  at  low  values  of  * . The  value  £ = 0.707  was  picked  as  the  best  for  this  loop. 

Since  this  value  of  £ is  at  the  lower  bound  for  stable  values  of  £ in  a mutual  ;/nc  system,  it  is 
obvious  that  a larger  value  will  have  to  be  picked.  One  should  increase  £ by  enough  to  ensure 
that  with  the  desired  hardware  implementation  one  cannot  have  changes  in  £ that  will  allow  the 
system  to  become  unstable.  In  addition,  larger  values  of  | are  desirable  because  the  transient 
response  of  a mutual  synchronization  is  more  oscillatory  than  that  of  a single  loop.  We  will 
tentatively  pick  £ = 1 being  a value  that  should  be  close  to  the  proper  value.  One  can  show 
that  the  steady-state  phase  error  for  this  value  of  £ is  only  about  37  percent  higher  than  that 
needed  for  £ = 0.707  (to  attenuate  a specified  jitter  frequency  by  a factor  of  10).  From 
Equation  (3. 1.2-6)  the  transform  of  the  phase  error  (at  £ = 1)  may  be  written  as 

$(s  + 2 (j  ) 

Els)  - j-  0.(s)  . (3. 2. 3-1) 
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The  phase  error  due  to  a step  in  frequency,  Ac u,  is  then  found  to  be 
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(3. 2. 3-2) 


It  is  easily  verified  that  the  phase  error  settles  to  within  1 percent  of  the  final  steady-state  error 

after  six  time  constants  (at  t - 6/cj  ^).  This  will  be  defined  as  the  acquisition  time.  If  one  tries  to 

attenuate  daily  jitter  by  a factor  of  10,  then  the  vaiue  of  CJ^  needed  at  £ = 1 is  found  from 

figure  3.  1 .2.2  to  be  cj  - 0.33  U).  = 2.41  x 10  ^ rad/s.  Unfortunately,  this  gives  an  acquisition 

n a 

time  of  about  69  hours.  This  would  be  an  undesirable  feature  in  a military  environment.  While  a 
particular  node  is  attempting  to  acquire,  other  nodes  are  also  locking  to  its  frequency  which  is 
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different  from  the  network  frequency.  Thus,  during  the  acquisition  period  its  frequency  will  tend 
to  pull  the  network  frequency  to  a different  value  which,  of  course,  is  undesirable.  Speeding  up 
the  acquisition  process  may  help  this  problem.  This  is  rather  important,  and  this  is  the  type  of 
behavior  one  must  be  concerned  with  in  evaluating  the  performance  in  a stress  environment  where 
link  and  node  outages  may  occur  frequently.  A better  choice  might  be  to  reduce  the  acquisition 

_3 

time  to  1 hour.  The  value  of  U)  required  is  u - 1 .67  x 10  rad/s.  The  steady-state  phase 

n n 

errors  for  these  two  loop  bandwidths  are  830  and  12  ps,  respectively,  for  a fractional  frequency 
offset  of  10 

The  above  discussion  approached  the  design  problem  principally  on  an  individual 
node  basis,  but  this  approach  is  not  really  adequate.  There  are  some  rather  significant  difference 
between  the  behavior  of  master-slave  and  mutual  sync  systems  which  lead  to  different  design  con- 
siderations. One  such  consideration  is  system  stability  discussed  in  the  previous  subsection. 
Another  difference  is  that  a mutual  synchronization  system  does  not  settle  down  to  a predeter- 
mined system  frequency.  The  final  system  frequency  is  determined  by  factors  such  as  the 
individual  VCO  center  frequencies,  changes  in  link  delays,  loop  parameters,  and  the  manner 
in  which  the  network  is  brought  into  operation.  Unlike  the  master-slave  system,  the  system 
frequency  of  a mutual  sync  system  can  be  permanently  changed  by  a change  in  a link  delay. 

One  can  see  this  from  the  following  example.  Suppose  we  have  a mutual  sync  system  where  the 
error  signal  applied  to  the  loop  filter  is  the  average  of  the  phase  errors  measured  on  all  links  to 
other  nodes.  If  a step  decrease  in  link  delay.  At,  occurred  on  each  link  simultaneously,  then 
all  phase  errors  measured  will  Instantaneously  increase  by  f A T . This  error  signal  will  be 
filtered  and  then  used  to  influence  the  VCO  frequency,  but  the  VCO's  at  all  nodes  will  change 
identically.  Since  all  VCO's  are  changing  in  the  same  direction,  we  can  assume  that  the 
increase  in  error  signal  will  be  maintained  at  fA  T , Thus,  one  can  determine  the  change  in 
VCO  frequency  by 


a K 

af(s)  = 7~r^  E(s)  * 


(3. 2. 3-3) 


where  the  error  signal  E< s)  - t Ar/s.  Then  the  change  in  VCO  frequency  at  all  nodes  are  the 
same  and  is  equal  to 


and  the  steady-state  fractional  change  in  system  frequency  due  to  the  change  in  delay  is 


= K At 
v 


(3. 2. 3-5) 


If  one  knows  the  maximum  amount  by  which  any  link  delay  may  change,  then  Equation  (3. 2. 3-5) 
can  be  used  to  determine  the  value  of  loop  gain  required  to  keep  the  fractional  change  in  system 
frequency  less  than  a predetermined  amount.  This  estimate  should  be  rathei  conservative,  though, 
since  Equation  (3. 2. 3-5)  was  computed  assuming  that  all  delays  change  simultaneously  by  the 
maximum  amount.  Obviously,  this  will  not  happen  in  a large  network.  Some  delays  will  be 
increasing  and  others  decreasing  resulting  in  a cancellation  effect  on  system  frequency.  For  the 

-3 

example  just  considered  with  the  1-hour  acquisition  time  (u>^  - 1 .67  x 10  ;ad/s)  we  find  that 
the  fractional  change  in  system  frequency  due  to  link  delay  variations  computed  from  Equation 
(3. 2. 3-5)  is  equal  to  8.3  x 10  for  a maximum  path  delay  variation  of  ±1  ps  (this  is  a good 
"ball  park"  estimate  for  the  links  on  the  DCS  with  the  exception  of  satellite  links).  The  cal- 
culated stability  of  the  system  frequency  is  adequate,  and  the  actual  stability  should  be  much 
better  than  this  bound.  This  trade-off  and  other  similar  trade-offs  are  discussed  in  a paper  by 


Moreland. 


Another  difference  with  a master-slave  system  is  that  the  error  signal  applied  to 


the  loop  filter  is  an  average  of  the  phase  errors  measured  with  all  of  the  links  at  that  node.  Thus, 

for  the  loop  with  = 1 .67  x 10  rad/s  the  maximum  error  signal  was  calculated  to  be  12  ps. 

The  worst-case  would  be  if  all  links  but  one  had  significant  phase  error  but  opposite  in  sign  from 

the  other  link  which  required  a large  phase  error  to  generate  the  proper  error  signal.  Thus,  the 

required  range  on  the  phase  detectors  (and  hence  the  buffer  sizes)  can  be  considerably  different 

from  the  maximum  error  signal  that  must  be  applied  to  the  loop  filter.  Unfortunately,  there  does 

133 

not  seem  to  be  any  way  to  calculate  the  maximum  phase  error  on  each  link.  Moreland  con- 
sidered this  problem  from  the  viewpoint  of  how  much  the  phases  of  nodal  clocks  will  change 
relative  to  each  other  due  strictly  to  changes  in  path  delays.  He  presented  the  dumbbell  con- 
figuration as  the  worst-case  topology  for  this  problem.  The  dumbbell  topology  has  the  nodes 
separated  into  two  groups  (each  group  being  highly  connected)  with  only  one  link  connecting  the 
two  groups.  He  claims  the  worst-case  occurs  when  all  links  connecting  one  of  the  groups  of 
nodes  experience  an  increase  in  delay  while  the  links  connecting  the  other  group  experience  a 
decrease  in  delay.  This  can  cause  a much  larger  change  in  phase  difference  between  the  clocks 
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on  either  end  of  the  single  link  connecting  the  two  groups  of  nodes.  However,  once  again  this  is 
a rather  extreme  case.  In  addition,  what  we  are  really  interested  in  is  the  peak  phase  error  that 
will  be  observed  on  any  link  relative  to  the  maximum  required  error  signal  (which  is  12  ps  in  our 
example).  This  peak  phase  error  is  probably  influenced  more  by  the  relative  offsets  in  VCO 
center  frequencies  at  each  node  than  by  path  delay  variations. 


Another  important  set  of  parameters  are  the  phase  detector  weighting  coefficients. 
Each  node  weights  the  phase  errors  measured  by  a set  of  relative  weightings.  Suppose  node  k 
has  N references  available.  Then  each  of  these  references  will  be  assigned  a relative  weight, 
the  weight  from  node  i being  denoted  by  w^..  At  any  given  time,  only  some  of  these  references 
may  be  available.  Of  course,  the  actual  weighting  of  those  references  should  reflect  only  the 
active  references.  Assuming  a subset  s of  these  references  are  active,  then  the  active  nodes 
should  be  weighted  by  a weight  of  the  form  w.  = ./ , where 


w - E w . (3. 2. 3-6) 

s 5 ki 


If  the  phase  error  measured  on  link  i is  denoted  by  0^.,  then  the  error  signal  applied  to  the 
loop  filter  is 


0 

e 


w 


(3. 2. 3-7) 


For  example,  one  possible  weighting  is  to  weight  all  references  equally.  In  this  case,  if  there 
are  n active  references,  each  will  be  weighted  by  the  factor  1/n.  Equal  weighting  has  one 
significant  advantage  in  that  it  provides  an  added  measure  of  survivability.  The  system  frequency 
is  not  heavily  influenced  by  a small  fraction  of  all  the  nodes.  One  might  try  to  obtain  a more 
stable  network  frequency  by  more  strongly  weighting  the  most  accurate  clocks  and  the  links  least 
affected  by  path  delay  variations.  This  does  work,  but  the  network  frequency  will  be  more 
strongly  perturbed  by  failures  of  these  more  strongly  weighted  nodes  and  links.  Determining  the 
best  trade  between  these  properties  is  somewhat  subjective  and  thus  difficult  to  do  analytically. 


3.2.4  S imuiation  Results 

Some  simulation  results  will  be  presented  in  this  section  to  illustrate  the  behavior 
of  a mutual  synchronization  network.  These  simulations  were  performed  using  the  dumbbell 
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topology,  and  most  used  the  topology  shown  in  Figure  3.2.4.  The  initial  simulations  were  done 
to  determine  acquisition  behavior  and  to  choose  The  parameters  discussed  in  the  previous 
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section  (£  = 1,  cj  = 1.67  x 10  rad/s)  were  chosen  as  a baseline.  Damping  varied  about 
n 

4 = 1 while  changing  u slightly  to  keep  the  -20  dB  point  on  the  closed  loop  response  at  the 

same  jitter  frequency  with  | = 1.  The  nominal  frequency  was  chosen  to  be  1.544  MHz  with  the 

“8  -8 

initial  VCO  fractional  frequency  offsets  being  -10  for  Nodes  1-4  and  +10  Hz  for  Nodes  5-8. 

Phase  error  weightings  were  equal.  It  was  found  that  increasing  £ increases  acquisition  time  and 
the  penalty  by  using  £ = 1 rather  than  £ near  0.707  is  about  30  percent  (steady-state  phase  errors 
will  also  be  higher  by  about  the  same  amount).  This  seems  like  a good  trade  to  obtain  better 
stability.  The  very  narrow  loops  prevent  the  network  from  exhibiting  any  tendency  to  oscillate. 

”3 

Thus,  the  values  of  £ = 1 and  u>  = 1.67  x 10  rad/s  were  chosen  for  the  remainder  of  the 
simulations.  It  was  found  that  for  £ = 1 the  initial  acquisition  required  11.7  hours  compared 
with  the  acquisition  time  for  a single  loop  of  only  1 hour.  One  would  expect  the  acquisition 
time  for  any  topology  to  be  several  times  that  of  a single  loop.  However,  this  topology  extends 
the  time  considerably  due  to  the  fact  that  the  nodes  on  the  right-hand  and  left-hand  sides  affect 
each  other  only  through  a single  link  4 5.  It  is  also  for  this  reason  that  the  steady-state  phase 
errors  build  up  in  an  unbalanced  manner.  The  worst  cases  are  at  Nodes  4 and  5.  At  Node  5 the 
phase  errors  with  respect  to  Nodes  4,  6,  and  7 are  -107,  36,  and  36  ps,  respectively,  giving  an 
average  phase  error  of  12  ps.  The  difference  between  the  magnitude  of  the  individual  phase 
errors  and  the  average  phase  error  indicates  the  difficulty  of  making  a good  estimate  of  the  size 
of  the  individual  phase  errors  (and  thus  buffers).  They  will  be  a function  of  both  network 
topology  which  is  always  subject  to  change  and  the  nominal  center  frequencies  of  the  VCO's  at 
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each  node.  Another  simulation  was  performed  to  observe  the  change  in  behavior  if  the  VCO 

nominal  center  frequencies  were  more  randomly  chosen.  The  VCO  fractional  frequency  offsets 

_8  _g 

were  changed  to  +10  at  Nodes  1 and  2 and  to  -10  at  Nodes  6 and  7.  As  a result,  the 
acquisition  time  was  improved  by  about  a factor  of  two,  and  the  peak  with  respect  to  other 
nodes  is  considerably  reduced.  Node  5 is  connected  to  three  other  nodes  (4,  6,  and  7)  each 
with  the  same  nominal  VCO  frequency  so  the  steady-state  phase  errors  with  respect  to  each 
of  those  nodes  is  virtually  identical  at  about  -12.3  ps.  However,  Node  4 is  connected  to  nodes 
with  differing  nominal  VCO  frequencies.  In  this  case  the  steady-state  phase  errors  with  respect 
to  Nodes  2,  3,  and  5 are  26.5-2.6,  and  12.3  ps,  respectively.  We  note  that  the  peak  phase 
error  does  not  occur  on  the  dumbbell  link. 

The  response  can  be  speeded  up  by  having  more  coupling  between  the  two  halves 
of  the  network.  This  can  be  done  through  a change  in  the  phase  error  weightings.  A simulation 
was  performed  identical  to  the  previous  simulation,  but  with  a different  phase  error  weighting 
scheme.  Nodes  2 and  3 weighted  the  phase  error  with  respect  to  Node  4 twice  that  of  Node  1. 
Likewise,  Nodes  6 and  7 did  the  same  in  the  other  half  of  the  network.  In  addition.  Nodes  4 
and  5 weight  the  phase  errors  measured  relative  to  each  other  twice  that  of  other  nodes  to  which 
they  are  connected.  This  allows  nodes  at  the  center  of  the  network  to  have  more  influence  than 
those  on  the  extremities.  Thus,  the  system  will  tend  to  behave  more  like  a master-slave  system. 
In  this  case  the  acquisition  time  and  the  phase  error  between  Nodes  4 and  5 were  reduced  by 
about  a factor  of  2 by  changing  the  phase  error  weightings.  Obviously,  further  improvements 
can  be  obtained  by  changing  the  weighting  coefficients  by  a factor  greater  than  2.  The  penalty 
one  pays  in  doing  this  is  that  the  system  will  be  strongly  influenced  by  a small  number  of  nodes 
and  links,  and  thus  outages  on  these  nodes  and  links  can  cause  severe  perturbations  in  nodal 
frequencies. 

Another  simulation  was  done  to  illustrate  the  impact  of  a link  dropout.  Each  node 

had  the  same  loop  parameters  and  VCO  offset  as  in  the  previous  simulations  with  equal  phase 

error  weightings.  After  acquisition  was  completed.  Links  5-6  drop  out.  Before  the  dropout 

the  system  frequency  had  a very  small  fractional  offset  (2  x 10  ^ ) from  the  nominal  system 

frequency.  After  the  transients  due  to  the  dropout  settle  out  (about  12  hours)  the  system  fre- 

-9  -9 

quency  had  a rather  small  negative  offset  (-10  ) for  a net  fractional  change  of  -1.2  x 10  in 

the  steady-state  system  frequency.  However,  the  transient  frequency  changes  were  an  order  of 
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magnitude  higher.  The  reason  is  that  when  Links  5-6  drop  out.  Node  5 then  obtains  its  average 
phase  error  from  only  Nodes  4 and  7 and  Node  6 obtains  its  phase  error  only  from  Node  8.  This 
causes  a large  step  change  in  the  average  phase  error  driving  the  loop  at  each  of  these  nodes. 

The  frequencies  at  these  nodes  are  most  severely  affected  by  this  transient.  At  Node  5 the 

-8 

maximum  value  of  the  transient  fractional  change  in  frequency  was  -1.4  x 10  while  at  Node  6, 

“8 

the  maximum  value  was  +1.4x10  . The  magnitude  of  these  transients  are  roughly  the  same  as 

those  encountered  during  initial  acquisition  except  at  the  nodes  far  removed  from  the  location 
of  the  outage . 

Simulations  were  also  performed  using  6 and  10  node  dumbbell  networks  to  determine 
the  effect  of  network  size.  As  one  might  expect,  we  found  that  the  acquisition  time  and  individual 
phase  errors  increased  with  N,  the  number  of  nodes.  For  the  frequency  offsets  we  used,  we  found 
that  the  steady-state  phase  error  between  the  two  dumbbell  nodes  behaved  as 

0 =31  N0,9  us  (3. 2. 4-1) 

e 

This  could  not  be  interpreted  as  a general  rule  since  the  phase  error  would  depend  on  both  topology 
and  phase  error  weighting  coefficients,  but  it  indicates  again  the  difficulty  of  estimating  the 
individual  phase  errors. 

The  most  severe  problem  observed  thus  far  is  that  link  and  node  outages  can  cause 
severe  perturbations  in  nodal  frequencies.  If  the  nominal  frequency  of  the  VCO  at  a node  is 
significantly  offset  from  the  network  frequency,  then  any  outage  that  causes  removal  of  one  of 
its  references  causes  a step  change  in  the  phase  error  driving  the  loop  filter.  This,  in  turn,  causes 
a significant  change  in  the  nodal  frequency  which  eventually  impacts  the  frequencies  at  all  other 
nodes.  However,  one  can  probably  reduce  the  magnitude  of  this  transient  by  not  allowing  such 
a large  step  change  in  phase  error  be  applied  to  the  loop  filter.  This  might  be  done  by  slowly 
forcing  the  phase  error  due  to  the  lost  link  to  approach  zero.  In  addition,  the  weighting 
coefficients  should  not  be  abruptly  changed  but  should  be  gradually  changed  over  a period  of 
time  to  the  desired  value.  The  time  frame  over  which  this  should  occur  has  not  been  determined 
at  this  point  but  should  be  a significant  fraction  of  the  network  settling  time  which  for  this 
example  was  about  12  hours.  A similar  approach  would  be  required  when  the  link  becomes 
operational  and  its  reference  is  used  again.  Of  course,  implied  in  this  approach  is  that  the 
phase  error  stored  due  to  the  lost  reference  is  accurate.  One  would  have  to  provide  measures 
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to  ensure  that  the  phase  error  did  not  drastically  change  just  prior  to  the  outage.  However,  even 
if  this  approach  could  virtually  eliminate  the  severe  transient,  then  one  would  still  have  the  per- 
manent change  in  system  frequency  caused  by  the  dropout.  The  magnitude  of  this  change  was 
about  an  order  of  magnitude  less  than  that  of  the  transient.  This  technique  above  for  reducing 
the  magnitude  of  the  transient  will  help  to  some  extent,  but  it  must  be  evaluated  further  before 
good  estimate  of  its  value  can  be  obtained. 

A summary  of  loop  parameters  for  the  mutual  sync  approach  is  given  in  Table  3.2.4. 

Table  3.2.4.  Loop  Parameters  for  the  Mutual  Sync  Approach 

£ = 1 , u = 1 . 67  x 1 0~^  rad/ s 
n 

_8 

For  a fractional  frequency  offset  of  1 x 10  , these  parameters  give  a : 

steady  state  phase  error  of  2 ps. 

3.3  Time  Reference  Distribution  Trade-Offs 

In  this  section  we  address  the  implementation  of  a time  reference  distribution 
approach  for  network  synchronization.  Conceptually,  the  approach  is  quite  similar  to  the 
master-slave  approach,  and  some  of  the  trade-offs  are  similar  to  those  made  for  the  master- 
slave  system.  A TRD-type  system  has  been  implemented  in  the  Canadian  Dataroute  and  is 
working  quite  wel  I . 
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3.3.1 


Time  Reference  Distribution  Block  Diagram 

A block  diagram  of  the  nodal  timing  subsystem  for  time  reference  distribution  is 
shown  in  Figure  3.3.1  . Again,  a dual  redundant  clock  system  is  shown  but  more  than  two  clocks 
can  be  utilized  to  increase  reliability  if  required. 
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Figure  3.3.1.  Time  Reference  Distribution  Timing  Subsystem  Block  Diagram 


An  error  measurement  device  is  shown  which  performs  the  function  of  measuring  the 
timing  error  (or  difference  in  precise  time)  between  the  local  clock  and  each  of  the  N incoming 
references.  This  is  done  using  the  following  double-ended  technique.  First,  each  reference 
carries  periodic  timing  marks,  e.g.,  the  framing  pulses  on  a multiplexed  data  stream.  Assume 
that  Node  b is  measuring  its  error  with  respect  to  Node  a.  The  time  of  the  clocks  at  these  nodes 
are  T^  and  T^,  and  the  link  delays  from  a to  b and  from  b to  a are  D^  and  D^,  respectively. 


Then  the  time  differences  measured  at  each  of  these  nodes  between  the  local  clock  and  the 
received  timing  marks  are 
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(3. 3. 1-1) 


AT 


T - 

a 


Tb  + 


D 


ba 


and 


+ D 


(3.3. 1-2) 


These  measured  timing  errors  contain  an  error  term  due  to  link  delays,  but  they  can  be  used  to 
find  the  actual  timing  difference  between  Nodes  a and  b.  Thus,  these  measured  quantities  are 
transferred  to  their  respective  TRD  control  elements  which  then  exchange  this  data.  This  is  done 
at  all  nodes  for  the  measured  timing  errors  on  all  incoming  references. 


The  TRD  control  element  is  also  shown  accepting  TRD  orderwire  information 

associated  with  each  of  these  references.  This  information  allows  the  processor  to  adaptively 

select  the  best  one  of  the  incoming  references.  Assume  that  Node  b has  selected  to  use  Node  u 

as  a reference  which,  in  turn,  is  referenced  to  the  network  master  through  another  path.  Then 

the  control  element  at  Node  b can  compute  its  actual  timing  error  relative  to  Node  a as 

(AT  - AT,  )/2.  A unique  feature  of  TRD  is  that  Node  b can  then  calculate  its  timing  error 
a b 

relative  to  the  network  master  if  it  knows  the  error  of  Node  a with  respect  to  the  master  (the 
measured  but  uncorrected  error  at  Node  a).  The  negative  of  the  timing  error  at  Node  b with 
respect  to  the  network  master  is  estimated  by 


AT,  = AT  + (AT  - AT,  )/2 
bm  am  a b 


= + To  - Tb  + (Dbo  - Dab>/2 


(3.3. 1-3) 


This  is  equal  to  the  desired  clock  correction  term  if  all  link  delays  are  equal  in  both  directions 
and  time  difference  measurements  are  perfect.  All  that  is  required  o implement  this  is  that  eachand 
every  pair  of  nodes  exchange  their  clock  correction  term  with  respect  to  the  master  (AT^).  The 
corrections  (AT,  ) are  processed  by  an  error  processor  which  determines  how  fact,  clock  correc- 
tion  should  be  made.  Numerous  approaches  for  implementing  the  error  processor  could  work.  We 
have  taken  the  approach  that  since  TRD  provides  closed  loop  control  of  the  nodal  frequency,  one 
can  design  the  error  processor  as  one  would  design  a loop  filter  in  a phase-locked  loop.  It  is 
chosen  to  behave  like  a very  narrowband  loop.  The  output  of  this  filter  provides  the  correction 
voltage  for  the  clock.  Any  type  of  clock  (either  atomic  or  quartz)  may  be  used  in  the  configuration 
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shown.  The  only  functions  that  would  be  different  would  be  different  filter  parameters  and  a 
different  technique  for  clock  correction.  With  an  atomic  clock  one  could  provide  the  clock 
correction  by  either  frequency  adjustment  of  the  clock  or  by  an  outboard  phase  shifter.  From  a 
performance  point-of-view  it  does  not  matter  which  approach  is  used;  they  accomplish  the  same 
result. 

The  rest  of  the  system  is  very  similar  to  the  master-slave  system  except  that  precise 
time  is  provided.  The  control  section  provides  operational  control  of  the  timing  subsystem  and 
reconfigures  the  redundant  hardware  in  case  of  failures. 

3.3.2  Time  Reference  Distribution  Tradeoffs 

The  time  reference  distribution  technique  is  basically  a double-ended  master-sic-  'e 
system  with  reference  to  the  absolute  phase  of  the  master.  There  are  several  unique  advantages 
to  this  approach.  The  double-ended  feature  eliminates  the  effect  of  path  delay  variations  on 
nodal  frequencies.  In  addition,  by  supplying  neighbors  with  information  about  the  measured  but 
uncorrected  error  in  the  local  clock  each  nodal  clock  is  corrected  based  on  an  estimated  error 
with  respect  to  the  network  master.  This  can  eliminate  the  effects  of  instabilities  in  clocks  in  the 
path  between  the  local  node  and  the  master.  Finally,  clock  correction  based  on  AT^m  allows  the 
difference  between  absolute  phases  of  the  local  clock  and  the  master  to  be  kept  small,  and  hence 
precise  time  can  be  distributed.  'Many  of  the  tradeoffs  one  must  make  in  selecting  a design  for 
such  a system  are  similar  to  those  discussed  in  Paragraph  3.1  for  the  master-slave  technique  while 
others  are  entirely  different.  Some  of  the  more  significant  areas  where  implementation  decisions 
must  be  made  are  the  following: 

• Adaptive  timing  distribution  network  organization  approach 

• Overhead  data  communication  requirements 

• Transmission  of  the  periodic  timing  mar k 

• Measurement  of  timing  error 

• Processing  of  timing  error  measurements  and  clock  correction 

• Clock  stabilities 
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We  have  not  had  time  in  this  study  to  perform  all  of  the  tradeoffs  associated  with 
these  areas  to  allow  a detailed  design  to  be  made.  We  have  only  performed  tradeoffs  in  those 
areas  which  would  significantly  impact  the  comparison  between  TRD  and  the  other  approaches. 

The  purpose  of  the  cdaptive  timing  distribution  network  organization  technique  is 

to  allow  each  node  to  select  its  best  reference  and  thus  provide  TRD  with  a high  degree  of 

survivability.  The  first  approach  for  accomplishing  this  was  proposed  by  Darwin  and  Prim. 

90S 

Modifications  of  this  technique  have  been  proposed  by  Stover.  The  Canadian  Dataroute 

92 

network  uses  a similar  organization  technique.  Thus,  this  problem  has  been  rather  thoroughly 

investigated,  and  we  did  not  see  the  need  for  additional  work.  We  have  selected  the  update 
counter  approach  recently  presented  by  Perreault.  Differing  approaches  to  the  problem  may 
organize  or  reorganize  with  different  degress  of  rapidity.  However,  the  effect  of  a change  in 
the  approach  on  performance  of  TRD  would  be  minor . 

Based  on  a rough  examination  of  the  overhead  requirements  associated  with  TRD 
they  appear  to  be  rather  small.  At  the  sampling  rates  envisioned,  a few  hundred  bit  per  second 
will  be  required  for  transmission  of  timing  error  measurements  and  identification  of  timing  marks. 
In  addition,  a few  hundred  bits  per  second  will  be  required  for  data  transfer  associated  with  the 
adaptive  reorganization  technique  (200  b/s  were  required  for  this  in  the  Canadian  Dataroute). 
Thus,  the  overhead  requirements  are  in  the  neighborhood  of  1 kb/s  for  each  reference  available 
at  a node. 

The  periodic  timing  marks  could  be  implemented  most  conveniently  by  utilizing  a 
multiplexer  framing  pattern  as  the  timing  mark  and  identifying  the  precise  time  of  each  mark 
through  data  transmitted  on  an  overhead  channel.  One  good  candidate  is  the  framing  pattern 
that  will  be  most  widely  available,  that  of  the  T1  data  stream.  One  bit  of  the  frame  sync  pattern 
is  transmitted  every  193  bits  (or  every  125  ms)  in  the  T1  data  stream.  If  better  resolution  in 
estimating  the  timing  error  is  desired,  a framing  pattern  from  a higher  level  multiplexer  could 
easily  be  used . 
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Timing  Error  Processing  and  Clock  Correction 


one 

Stover  has  suggested  that  the  timing  error  samples  (AT^)  as  given  in 

Equation  (3. 3. 1-3)  measured  with  respect  to  the  master  clock  be  processed  and  used  to  correct 

the  phase  of  the  local  clock.  As  mentioned  previously,  we  have  chosen  to  implement  the  error 

processor  as  one  would  implement  a loop  filter.  Although  we  believe  our  approach  to  be  a very 

good  one,  we  are  not  arguing  that  it  is  optimum.  One  could  use  other  approaches  for  error 

processing  that  would  work  very  well.  The  filter  we  have  chosen  has  characteristics  very  similar 

to  that  we  have  chosen  for  the  master-slave  approach.  This  approach  allows  a frequency  offset 

to  be  tracked  with  zero  steady-state  timing  error,  and  a linear  frequency  drift  to  be  tracked  with 

2 

a constant  error  proportional  to  1 /u  . In  establishing  the  parameters  for  this  technique,  we  will 

n 

use  much  of  the  analysis  developed  in  Paragraph  3.1  . 

The  following  analysis  assumes  an  analog  implementation.  Actually,  the  timing 
error  samples  will  be  token  at  discrete  intervals,  but  they  will  be  taken  so  frequently  relative  to 
the  loop  time  constant  that  the  behavior  will  closely  approximate  that  of  an  analog  system.  After 
establishing  the  desired  parameters,  we  will  determine  how  often  phase  error  samples  must  be 
taken . 

The  error  processing  filter  we  have  selected  is  shown  in  Figure  3. 3. 2.1 . It  is  just 
an  integral  plus  proportional  filter.  If  timing  errors  AT,  measured  in  NS  are  fed  to  the  filter,  a 
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Figure  3.3.2. 1 . Error  Processing  Filter  for  TRD 


correction  term  C in  n S/s  appears  on  the  output.  As  indicated  the  proportional  term  performs  the 
short  term  phase  correction  while  the  integral  term  performs  the  correction  for  a frequency  offset 
between  the  local  clock  and  the  master.  The  time  constants  for  these  two  correction  terms  can  be 
made  considerably  different  by  choosing  a <<  k (i.e.,  selecting  a large  value  of  £ ). 

We  have  determined  two  sets  of  filter  parameters,  one  set  for  cesium  clocks  and 
another  set  for  quartz  clocks.  The  accuracy  of  cesium  clocks  is  typically  I part  in  10  . The 

network  master  will  be  cesium,  and,  therefore,  the  nominal  frequency  of  any  other  cesium  clock 
in  the  network  will  be  offset  from  the  frequency  of  the  master  by  at  most  2 parts  in  10  . Thus, 

the  inherent  high  degree  of  accuracy  of  cesium  clocks  ensures  that  they  will  not  have  to  acquire 
from  large  frequency  offsets.  Since  these  clocks  are  highly  stable,  the  parameters  will  be  chosen 
such  that  the  nodal  frequency  will  not  deviate  much  from  the  frequency  of  the  network  master 
(by  no  more  than  a few  parts  in  10^). 

We  are  principally  concerned  with  the  behavior  of  this  system  acquisition  from 

phase  and  frequency  offsets.  This  behavior  is  identical  to  that  of  a type  two  phase-locked  loop. 

The  peak  frequency  change,  Af^,  due  to  a step  change  in  phase  is  given  by  Equation  (3.1.2-17) 

Step  changes  in  phase  can  be  introduced  by  a change  to  another  reference  link,  but  these  should 

be  much  less  than  1 ps.  Rapid  changes  in  link  asymmetries  and  measurement  errors  can  also  cause 

phase  steps  at  the  input  to  the  loop,  but  the  magnitude  of  these  should  be  much  less  than  that 

caused  by  c change  in  reference.  Assume,  for  the  moment,  that  the  loop  parameters  used  in  the 

master-slave  system  are  chosen  (£  = 4 and  a > —5.6=  10^  rad/s).  Then 

n 

K = 4.5  x 10  4 s 1 and  a = 7 x 10  ^ s ^ 

The  peak  frequency  change  due  to  a 1 ps  phase  step  is  a fractional  frequency  change  of  KAT,  or 
4.5  x 10  .It  would  be  desirable  to  reduce  this  frequency  swing  by  about  an  order  of  magni- 
tude. This  can  be  done  by  reducing  the  product  K by  an  order  of  magnitude.  The  large  value  of 
£ was  chosen  in  the  master-slave  system  to  prevent  a large  accumulation  of  jitter  due  to  path  delay 
variations.  Since  these  variations  are  removed  by  TRD  we  do  not  need  to  make  £ so  large.  From 
Figure  3.1  .2.2  we  see  that  if  ? = 2 is  chosen,  we  can  still  keep  the  peak  value  of  | H ftu)  j < 1.05. 

Thus,  £ = 2 can  be  chosen  without  causing  much  jitter  accumulation.  Assuming  this  value  of  £ is 

-5 

chosen,  then  by  reducing  u)  by  a factor  of  5 to  1 .12  x 10  rad/s  the  fractional  frequency 
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instability  caused  by  a 1 ps  phase  step  can  be  reduced  by  a factor  of  10  to  4.5  x 10  . Of 

course,  smaller  phase  steps  will  cause  proportionately  smaller  swings  in  frequency. 

To  see  if  these  parameters  were  entirely  satisfactory,  we  also  checked  the  peak 
phase  error  that  occurs  during  acquisition  from  a frequency  offset.  The  peak  phase  error  for  the 
overdamped  loop  is  given  by  Equation  (3. 1 .2-16).  For  a large  value  of  damping,  this  is 
approximately 


AT  peak  « Af/Kf 


(3. 3. 2-1) 


The  loop  should  not  have  to  acquire  from  any  fractional  frequency  offset  greater  than 
2 x 10  ^ since  the  master  will  also  be  cesium.  Then  from  Equation  (3.3.2-')  the  peak  timing 
error  during  acquisition  is 

AT  peak  = 0.44  ps 

This  is  well  within  the  goal  of  keeping  all  nodal  clocks  within  ±2  ps  of  the  network  master. 

TRD  can  also  be  used  with  quartz  clocks,  but  the  lower  stability  of  the  quartz 
clocks  will  force  a change  in  filter  parameters.  The  principal  problem  is  frequency  drift.  Assume 
for  the  purposes  of  this  discussion  that  the  crystal  VCO  has  a drift  rate  of  1 part  in  10^  per  day. 
For  the  loop  parameters  found  above  for  the  cesium  clock,  the  steady-state  phase  error  caused  by 
the  drift  is  found  from  Equation  (3.1.2-21)  to  be  6 = 9.23  ps  which  is  unsatisfactory.  This 

steady-state  error  can  be  reduced  by  a factor  of  100  by  increasing  by  a factor  of  10  to 

u>  = 1 .12  x 10  ^ rad/s.  With  this  value  of  ai  we  find  that  the  peak  frequency  swinq  due  to  a 

n n -10 
one  cycle  phase  step  has  increased  by  a factor  of  10  to  a fractional  instability  of  2.9  x 10 

over  that  obtained  with  the  cesium  clock.  This  is  still  good  frequency  stability.  However,  the 

g 

crystal  clocks  may  have  to  acquire  from  initial  frequency  offsets  as  high  as  a few  parts  in  10  . 

From  Equation  (3.1.2-16)  we  find  that  during  acquisition  from  a fractional  frequency  offset  of 
2x10  the  peak  phase  error  with  these  loop  parameters  is  39  ps  which  is  too  high.  A wider 
bandwidth  fast  acquisition  mode  for  crystal  clocks  is  desirable.  The  same  strategy  used  with  the 
master-slave  approach  should  be  adequate  for  TRD.  If  the  phase  error  gets  larger  than  2 ps  then 
the  loop  will  switch  to  an  acquisition  bandwidth  with  loop  parameters  £ = 0.707  and  u ^ =0.007 
rad/s.  The  initial  phase  offset  on  the  acquisition  loop  will  cause  a peak  frequency  swing  of 

-9 

6.4  x 10  . Close  lock  will  be  achieved  in  30  minutes. 


3-38 


Table  3. 3. 2-1.  TRD  Error  Processing  Filter  Parameters 


Cesium  Clocks 

Only  tracking  parameters  are  used 
w - 1.12  x 10  rad/s,  £ = 2 

These  give  K = 4.5  x 10  ^ s ^ and  a = 2.8  x 10  ^ s ^ . 

This  gives  a peak  frequency  error  of  4.5  x 10  ^ when  a step  of  1 ps  of  phase  offset 
is  applied.  They  give  a peak  phase  error  of  0.44  ps  when  a fractional  frequency  step  of 
2x10  ^ is  applied  . 

Quartz  Crystal  Clocks 

Tracking  = 1 .12  x 10  rad/s,  $ = 2.  This  results  in  K = 4.5  x 10  ^ s ^ 
and  a - 2.8  x 10  s . For  drift  rate  of  1 x 10  ^ per  day,  these  parameters  provide  a 
steady-state  phase  error  of  0.09  ps. 

Fast  Acquisition  u>  - 0.007  rad/s,  £ = 0.707.  This  results  in  K = 0.001  s ^ and 

-1  n 

a = 0.005  s 

These  parameters  were  used  in  the  simulations  for  comparisons  with  the  other 
techniques.  It  is  possible  that  one  could  make  several  improvements  on  this  approach.  For 
example,  Stover  has  indicated  that  it  will  probably  be  desirable  for  a node  initially 

entering  the  network  to  make  a step  phase  correction  to  its  clock.  This  can  easily  be  done 
and  will  avoid  the  transient  associated  with  acquisition  from  a phase  offset.  In  addition,  it 
may  be  desirable  to  use  a much  longer  time  constant  on  the  integrator  for  cesium  clocks 
(decrease  a).  This  will  have  the  effect  of  making  frequency  offset  corrections  as  a result  of  a 
much  longer  averaging  time.  An  averaging  time  of  several  weeks  could  be  used.  Then  if  the 
reference  is  lost,  the  clock  corrections  will  continue  to  be  made  by  the  integrator  output  and 
they  will  be  the  result  of  estimating  the  frequency  offset  between  the  local  clock  and  the  net- 
work master  over  a several  week  period.  Of  course,  this  change  in  the  value  of  a can  be  made 
without  changing  the  value  of  K so  the  short-term  phase  corrections  would  not  be  changed  much. 
All  that  has  to  be  done  is  to  increase  £ and  decrease  u>  by  the  same  factor. 
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These  loops  are  digital  loops  and  thus  the  sampling  rate  must  be  chosen  higri  enough 
relative  to  the  filter  time  constants.  The  shortest  time  constant  in  the  system  is  that  of  the 
acquisition  mode  for  crystal  clocks.  The  time  constant  is  jj-  = 202  s.  Six  to  eight  phase 
error  samples  per  time  constant  is  adequate  to  approximate  analog  performance.  However, 
assume  that  we  oversample  and  input  a sample  every  20  s to  the  loop  filter.  Each  sample  should 
be  as  accurate  as  possible,  so  to  achieve  greater  accuracy  it  should  be  an  average  of  many 
samples . It  is  we  1 1 known  that  if  one  is  trying  to  estimate  a quantity  from  independent  noisy  measure- 
ments that  the  standard  deviation  of  the  average  value  of  n measurements  is  1 /[ n times  the 
standard  deviation  of  a single  measurement.  If  we  assume  that  each  timing  error  measurement 
can  be  made  to  within  100  ns,  then  by  making  10  measurements  per  second  and  averaging  200 
such  measurements  every  20  s to  obtain  an  average  timing  error,  the  measurement  error  can  be 
reduced  to  7 ns.  While  these  numbers  may  not  reflect  the  system  one  might  design,  they  indicate 
the  tradeoff  between  measurement  error  and  sampling  rate.  If  measurement  error  is  greater,  then 
one  can  increase  the  number  of  samples/second  to  obtain  the  same  accuracy  of  phase  error 
samples  at  the  input  of  the  loop  filter.  The  penalty  one  would  incur  by  increasing  the  sampling 
rate  would  be  increased  overhead  data  transmission. 


3.3.3  Effect  of  Link  Asymmetries 

One  of  the  significant  advantages  of  TRD  is  that  being  a double-ended  system,  it 
removes  the  effect  of  path  delay  variations  on  nodal  frequencies.  However,  in  doing  this,  it 
does  become  sensitive  to  asymmetries  in  reciprocal  path  delays.  The  link  asymmetries  may  be 
classified  into  three  groups. 

• Fixed  asymmetries  in  equipment  delays 

• Variable  asymmetries  in  equipment  delays 

• Variable  asymmetries  in  path  delays  of  transmission  links 

The  phase  error  measured  at  any  node  with  respect  to  the  master  is  given  by 
Equation  (3.3. 1-3).  The  effect  of  an  asymmetry  is  the  change  the  phase  error  estimates  by  the 
quantity  (D^  - D^)/2.  Thus,  the  effect  on  TRD  of  a change  in  link  asymmetry  by  a certain 
amount  is  identical  to  the  effect  on  a master-slave  system  (with  the  same  loop  parameters)  of  a 
change  in  path  delay  of  the  same  magnitude. 
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Fixed  asymmetries  in  equipment  delays  for  the  two  directions  of  a link  are  due  to 
slightly  different  signal  transit  time  through  hardware  such  as  transmitters,  receivers,  multiplexers, 
and  demultiplexers.  One  should  consider  these  effects  when  deciding  where  to  make  the  phase 
error  measurement.  Thus,  making  this  measurement  at  the  highest  multiplexing  level  may  be  best. 

Although  they  are  typically  rather  small,  fixed  asymmetries  I t equipment  delays 
will  cause  a one-time  frequency  transient  during  acquisition  as  a result  or  caving  to  acquire  a 
phase  offset.  This  will  cause  a permanent  error  in  the  piecise  time  kept  at  the  node  equal  to 
half  the  magnitude  of  the  asymmetry,  but  if  will  not  affect  the  nodal  frequency  after  the  initial 
transient.  However,  if  the  node  needs  to  acquire  a reference  on  another  link,  ' nen  the  dirferent 
asymmetry  will  cause  the  loop  to  have  acquired  from  another  phase  offset  wt  h again  causes  a 
one-time  frequency  transient.  These  effects  can  be  eliminated  by  measurement  of  the  asymmetries 
and  allowing  the  phase  error  estimator  to  account  for  them  in  calculating  the  phase  error. 

To  optimize  performance  variable  asymmetries  in  equipment  delays  should  be 
accounted  for  in  estimating  phase  error.  One  case  of  significance  is  when  a TRD  node  is 
referenced  to  another  TRD  node  through  an  intermediate  node  w hich  does  MUX-DEMUX  opera- 
tions but  does  not  have  a TRD  nodal  timing  subsystem.  Assume,  foi  the  moment,  that  the 
framing  oit  on  a T1  data  stream  is  used  as  the  timing  mark.  Then  the  delay  through  the  node  in 
the  two  directions  may  be  unequal  due  to  different  buffer  delays  in  the  two  directions.  One  can 
envision  several  approaches  for  handling  this  problem.  These  buffer  delays  should  be  measured 
periodically  at  the  intermediate  nodes  and  reporled  via  an  overhead  channel  to  the  other  nodes 
concerned  so  that  corrections  to  the  phase  error  estimate  can  be  made.  Another  approach, 
would  be  to  provide  elastic  buffers  in  the-  transmit  channel  and  provide  the  necessary  control 
for  ensuring  rhar  rhe  butfc  ill  in  each  transmit  channel  is  such  that  the  delays  in  both 
directions  of  all  TTD  pat’  *■  occ  rhe  node  are  equal. 

The  magnitude  of  ua^iabie  asymmetries  in  path  delays  of  transmission  links  should 
be  much  less  than  the  magnitude  of  one-way  path  delay  variations.  For  a satellite  link,  there 
is  considerable  variation  in  total  path  delay,  but  since  the  link  is  LOS,  there  will  be  no 
asymmetry  in  path  de’ay.  A microwave  link  is  also  LOS,  and  there  will  be  very  little  variation 
In  total  path  delay  as  well  as  link  asymmetry.  Cable  is  subject  to  path  delay  variations  due  to 
temperature  changes.  However,  the  delay  is  affected  in  the  same  manner  in  both  directions 
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and  there  are  no  changes  in  link  asymmetry.  Foi  each  of  these  types  of  links,  the  TRD  system 
should  perform  much  better  than  the  master-slave  system.  In  a link  with  scattered  propagation 
such  as  a tropo  asymmetries  may  occur,  but  no  measurments  of  reciprocal  link  delays  for  tropo 
have  been  made.  However,  consider  a link  where  the  path  delay  of  a link  can  vary  by  ±AD 
about  the  nominal  delay,  D.  The  worst-case  asymmetry  occur s when  the  path  delay  in  one 
direction  is  D ^ AD  while  the  delay  in  the  other  direction  is  D - AD.  The  total  asymmetry  is 
2 AD.  The  total  asymmetry  is  2-D  and  contribution  of  this  asymmetry  to  measurement  error  is 
AD.  Note  that  this  will  affect  the  nodal  frequency  for  TRD  in  the  same  manner  as  a change  in 
path  delay  of  AD  v/ould  affect  the  master-slave  system.  However,  to  have  this  occur  would 
require  that  the  delay  in  one  direction  of  the  link  change  by  + AD  while  the  delay  in  the  other 
direction  of  the  link  changes  by  - AD.  This  situation  is  much  less  likely  to  occur  than  having 
a single  link  change  by  AD  which  would  affect  a master-slave  system  in  the  same  manner. 

In  summary,  fixed  asymmetries  which  are  not  calibrated  out  cause  a steady-state 
error  in  the  precise  time  of  nodal  clocks.  These  errors  can  cause  transients  in  nodal  frequencies 
during  acquisition  but  no  perturbations  thereafter.  Variable  asymmetries  which  are  not  calibrated 
out  can  cause  perturbations  in  nodal  frequencies  in  much  the  same  way  that  path  delay  variations 
can  cause  perturbations  in  nodal  frequencies  in  u master-slave  or  mutual  sync  systems.  However, 
since  the  magnitude  of  the  asymmetries  is  much  less  than  the  magnitude  of  path  delay  variations, 
the  resulting  perturbations  are  less. 
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DESIRABLE  CHARACTERISTICS 


A number  of  characteristics  have  been  identified  that  if  possessed  by  a timing  sub- 
system would  be  considered  advantageous  from  a system  viewpoint.  These  desirable  character- 
istics, although  not  necessary  for  the  provision  of  a minimal  level  of  performance,  do  enhance 
the  overall  utility  of  the  timing  subsystem.  The  desirable  characteristics  are  survivability, 
limited  error  propagation,  compatibility  with  other  global  timing  subsystems,  precise  time 
availability,  stability,  monitorability,  minimum  susceptibility,  flexibility,  interoperability  and 
maximum  availability.  In  the  paragraphs  below,  each  of  these  characteristics  is  defined  and  is 
justified  as  being  important  in  the  overall  subsystem  evaluation.  In  a subsequent  section,  each  of 
the  alternative  techniques  is  evaluated  with  respect  to  the  degree  to  which  it  provides  these 
desirable  characteristics. 

4.1  Survivability 

Survivability  indicates  the  degree  to  which  the  timing  subsystem  continues  to  per- 
form its  function  (to  provide  network  timing)  during  periods  of  stress.  Fallback  modes  of  operation 
shall  be  provided  so  that  the  timing  subsystem  itself  will  not  cause  significant  degradation  in 
overall  DCS  network  capability  during  these  periods. 

There  are  two  main  reasons  why  this  characteristic  is  important: 

1 . One  of  the  primary  goals  of  the  DCS  is  to  provide  dependable  digital  communi- 
cations for  its  users  during  periods  of  network  stress. 

2.  The  network  timing  subsystem  must  be  operational  to  provide  acceptable 
digital  communication  during  such  periods. 

Of  course,  the  reason  that  network  synchronization  is  necessary  is  that  the  slip 
rate  that  two  nodes  will  experience  in  communicating  is  directly  proportional  to  the  frequency 
offset  between  their  nodal  clocks.  If  this  offset  increases  in  a stressed  environment,  then  bit 
slips  increase  which  lead  to  misaligned  frames  in  multiplexers  and  switches  and  a need  to 
resynchronize  cryptos.  If  the  slip  rate  becomes  excessive,  data  traffic  will  experience  an 
unacceptably  high  error  rate  and  the  quality  of  voice  transmissions  will  degrade  significantly. 
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Error  Propagation 

It  is  desirable  to  prevent  disturbances  (such  as  stresses  or  path  delay  variations)  in 
one  part  of  the  network  from  propagating  and  influencing  the  phase  of  nodal  clocks  in  other  parts 
of  the  network . 

This  is  an  important  characteristic  because  in  preventing  the  propagation  of  per- 
turbations in  nodal  frequencies  from  node  to  node  throughout  the  network  one  maximizes  the 
stability  of  each  nodal  clock  and  minimizes  the  buffer  size  required  to  achieve  a desired  slip 
rate.  The  parameter  that  will  be  used  as  a measure  of  error  propagation  is  the  offset  in  phase  of 
each  nodal  clock  from  their  steady  state  values  as  a result  of  certain  network  stresses. 

4.3  Compatibility  With  Other  Global  Timing  Subsystems 

This  characteristic  refers  to  the  degree  with  which  the  DCS  network  frequency  and/ 
or  precise  time  agrees  with  that  provided  by  other  global  Navigation/Timing  Systems  such  as 
Loran-C,  Omega  and  GPS. 

There  are  two  principle  reasons  why  this  characteristic  is  important: 

1 . The  navigation  system  timing  could  be  used  as  an  external  reference  in  a 
fallback  mode  thus  increasing  the  survivability  of  the  DCS  timing  system. 

2.  The  precise  time  or  frequency  provided  by  the  DCS  timing  subsystem  would  be 
available  to  the  navigation  system  to  increase  its  reliability  or  capability. 

4.4  Precise  Time  Availability 

Precise  time  can  be  defined  as  the  correct  absolute  time  as  kept  by  a standards 
laboratory  such  as  the  USNO.  The  timing  subsystem  could  be  designed  to  acquire  and  dissemi- 
nate precise  time  from  the  USNO  within  some  margin  of  error.  The  availability  of  and  accuracy 
of  the  precise  time  thus  distributed  throughout  the  network  is  a characteristic  by  which  we  shall 
evaluate  the  timing  subsystems. 

There  are  four  main  reasons  why  this  characteristic  is  important: 
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1 . The  network  is  automatically  referenced  to  UTC  and  has  the  advantages  of 
better  interoperability  with  other  networks  referenced  to  UTC  and  compati- 
bility with  other  global  timing  systems. 

2.  Precise  time  is  useful  in  assisting  synchronization  of  spread  spectrum  equipment 
and  in  providing  synchronization  for  TDMA  networks. 

3.  Precise  time  would  be  more  widely  available  to  those  users  of  precise  time 
within  the  government.  This  will  simplify  their  problems  of  updating  their 
precise  time  standards. 

4.  Navigation  systems  could  use  the  DCS  timing  to  increase  their  reliability  and 
capability. 

4.5  Stability 

The  DCS  timing  subsystem  must  be  a stable  system  (as  a feedback  control  system). 

This  is  important  because  if  the  system  were  not  stable,  then  a number  of  bad 
things  could  happen  such  as  nodal  clocks  being  continually  pulled  off  in  one  direction  or  simply 
being  unable  to  maintain  network  synchronism.  The  result  of  such  an  occurrence  would  be 
buffer  overflows  and  therefore,  continual  bit  slips. 

The  mutual  synchronization  approach  has  closed  feedback  loops  and  stability  is 
of  concern  here.  However,  stability  can  be  nominally  guaranteed  by  proper  selection  of  the 
loop  filter  in  the  phase-locked  loops.  The  question  of  whether  stability  is  a problem  under 
certain  network  stresses  for  this  technique  or  whether  it  is  a problem  for  the  other  approaches 
which  transmit  control  information  from  node  to  node  (for  example,  to  modify  the  timing  dis- 
tribution chain)  does  need  to  be  examined.  We  must  prove  the  stability  of  each  alternative. 

If  there  is  any  possibility  of  instability,  we  must  determine  what  can  be  done  to  ensure  that 
this  does  not  happen. 

4.6  Monitorability 

Monitorability  is  the  level  to  which  the  timing  subsystem  can  provide  data  used 
in  assessing  its  own  performance.  It  is  apparent  that  monitorability  is  valuable  at  least  to  the 
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extent  that  an  option  such  as  a redundant  piece  of  equipment  or  an  alternate  link  is  available 
and  can  be  utilized  as  a result  of  decisions  made  on  the  basis  of  monitored  information. 


Monitorability  is  a significant  characteristic  because  it  can  lead  to  greater  availa- 
bility. If  potential  failures  can  be  identified  prior  to  failures  actually  occurring,  these  situations 
can  be  remedied  on  an  off-line  basis.  The  result  is  therefore  both  an  increase  in  MTBF  and  a 
reduction  in  MTTR  and  therefore  a greater  availability. 

4.7  Susceptibility 

Susceptibility  is  the  degree  to  which  the  timing  subsystem  may  be  spoofed,  jammed 
or  disrupted  by  an  enemy  via  electronic  means  as  well  as  disrupted  due  to  natural  phenomena. 

Because  of  the  critical  nature  of  the  DCS  mission,  it  is  apparent  that  it  could  be 
a focal  point  of  enemy  action  during  conflict.  Therefore,  susceptibility  is  a key  characteristic. 
Any  timing  subsystem  that  by  its  nature  increases  the  susceptibility  of  the  DCS  as  a whole  repre- 
sents a significant  handicap.  Slip  rate  will  again  be  the  measurable  quantity  in  evaluating 
susceptibility.  The  evaluation  consists  of  measuring  the  rate  of  accumulation  of  phase  error 
between  the  nodal  clocks  (slip  rate  is  directly  proportional  to  the  rate  of  accumulation  of  phase 
error  between  the  nodal  clocks  at  two  nodes  and  inversely  proportional  to  the  buffer  size)  under 
a variety  of  possible  stress  scenarios. 

4.8  Flexibility 

Flexibility  is  the  degree  to  which  the  timing  subsystem  is  compatible  with  the 
orderly  implementation,  growth  and  extension  of  the  future  synchronous  switched  digital  DCS. 

Flexibility  is  an  extremely  important  characteristic  as  a result  of  both  the  time 
frame  for  eventual  utilization  and  the  transition  period  during  which  the  system  must  operate 
effectively.  The  timing  subsystem  itself  is  aimed  at  a system  to  be  operational  in  the  1 980's 
and  until  then,  daily  traffic  consisting  of  voice  and  data  must  be  accommodated  by  the  system. 


i 4.9  Interoperability 

t 

Interoperability  is  the  degree  to  which  the  timing  subsystem  influences  the  traffic 
interface  between  DCS  and  other  military  and  commercial  communications  systems. 

Interoperability  is  a significant  characteristic  since  a portion  of  the  DCS  traffic 
will  traverse  both  DCS  as  well  as  other  systems  (i.e.,  Tri-Tac)  between  origination  point  and 
destination  point.  Furthermore,  the  DCS  consists  of  both  leased  facilities  (CONUS)  and  owned 
facilities  (OCONUS).  The  degree  of  design  independence  and  control  is  far  greater  in  the  owned 
facilities  than  in  the  leased  facilities.  Thus,  there  will  be  a major  interface  within  the  DCS 
itself. 

4.10  Avai  labi  lity 

, Availability  is  the  quotient  of  mean-time-before-failure  (MTFB)  and  the  sum  of 

mean-time-before-failure  (MTBF)  and  the  mean-time-to-repair  (MTTR).  Availability  is  a signi- 
, ficant  characteristic  for  all  major  systems  and  its  importance  is  apparent  for  the  timing  subsystem 

by  nature  of  the  mission  of  the  DCS. 
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NETWORK  CONFIGURATIONS 


This  section  delineates  the  network  configurations  that  will  serve  as  the  test  beds 
in  which  the  performance  of  the  alternative  timing  techniques  will  be  evaluated.  Each  network 
configuration  has  been  chosen  to  combine  realism  with  relevance.  Realism  meaning  that  the 
configuration  represents  an  interconnection  of  nodes  that  is  representative  of  a world  wide 
communications  network  or  portions  thereof.  Relevance  being  defined  as  aiding  in  the  selection 
process  between  alternative  timing  techniques.  That  is,  the  network  configurations  have  been 
chosen  with  the  purpose  of  highlighting  strengths  and  weaknesses  of  the  various  timing 
techniques. 

To  completely  specify  the  network  configuration  for  the  simulation  the  following 
parameters  have  to  be  defined:  the  connectivity  arrangement  between  the  nodes,  the  distance 
between  each  pair  of  connected  nodes,  the  bandwidth  of  each  link,  the  type  of  transmission 
media  used  between  each  pair  of  connected  nodes  and  an  indication  as  to  which  nodes  are 
external  to  the  DCS  (and  therefore  links  to  these  nodes  constitute  interface  links). 

The  method  that  has  been  selected  to  specify  these  configurations  is  to  initially 
choose  a set  of  connectivity  patterns.  These  patterns  are  each  appropriate  combinations  of  the 
subpatterns  (ring,  fully  connected,  star)  discussed  in  Section  11.0.  The  connectivity  patterns 
that  have  been  chosen  are  shown  in  Figures  5.1-1  through  5.1-5  and  are  described  in  following 
paragraphs. 


5 . 1 Network  Connectivities 

a . Dumbbell  Node  Configurations 

The  Dumbbell  Node  Configuration  (Figure  5. 1-1)  is  so  called  because  of  its 
peculiar  shape.  It  consists  of  two  relatively  congested  communications  areas  interconnected 
by  a relatively  small  number  of  links.  In  Figure  5.1-1,  the  left-hand  grouping  of  nodes  is 
primarily  interconnected  to  the  right  hand  group  of  nodes  through  Nodes  14  and  16.  A second- 
ary interconnection  between  the  two  groupings  is  made  through  Nodes  15  and  16.  This  partic- 
ular type  configuration  is  primarily  found  where  large  bodies  of  water  or  uninhabited  land  are 
crossed  by  communications  systems.  Across  a body  of  water,  for  instance,  the  interconnecting 
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links  would  be  satellite  or  undersea  cable.  Across  a body  of  land,  it  may  be  a microwave 
link  or  tropo  or  landline  in  place  of  submarine  cable.  The  right-hand  congested  network  is 
typical  of  a lattice-type  network  while  the  left-hand  network  is  more  random. 

b.  Meandering  Node  Configuration 

The  Meandering  Node  Configuration  (Figure  5.1-2)  is  so  named  because 
of  the  backbone  node  path  consisting  of  Nodes  5 , 8,  14,  18  , 20,  23  , 24  and  29  are  inter- 
connected to  a meandering  path  similar  to  a river.  Each  backbone  node  has  clustered  around 
it  a number  of  concentrated  nodes.  The  two  extremes  of  the  meandering  configuration  have 
little  in  the  way  of  direct  communications  paths  between  them  except  the  satellite.  This 
configuration  shows  the  availability  of  a satellite  communications  path  connecting,  for 
instance.  Nodes  8,  18  and  29. 

c.  Lattice  Node  Configuration 

The  Lattice  Node  Configuration  (Figure  5.1-3)  is  typical  of  a highly 
developed  and  congested  node  configuration  such  as  would  exist  in  CONUS  or  in  some  of  the 
highly  congested  European  area.  It  illustrates  a great  many  interconnecting  paths  available 
between  nodes.  This  configuration  has  a fluency  of  alternate  routing  paths  that  are  beneficial 
in  peak  traffic  periods  and  to  circumvent  equipment  outage.  In  this  particular  figure,  the 
large  majority  of  nodes  can  easily  be  made  to  belong  to  the  highest  level  in  the  nodal  hier- 
archy. This  results  from  the  equivalent  amount  of  connectivity  for  these  nodes. 

d.  Ladder  Node  Configuration 

The  Ladder  Node  Configuration  (Figure  5.1-4)  gets  it  name  from  the  fact 
that  sublevel  nodes  are  arranged  in  the  form  of  a ladder.  To  illustrate  this,  look  at  Nodes  23, 
27,  29,  24,  28  and  30.  These  form  the  sides  of  the  ladder  and  the  interconnections  between 
the  nodes  such  as  27  to  18  and  29  to  30,  form  the  rungs  of  a ladder.  There  are  two  ladder 
configurations  shown  on  this  diagram;  one  on  the  left  side  consisting  of  Nodes  1,  2,  3,  4,  5, 
and  6 and  the  one  previously  discussed  on  the  right.  The  remaining  nodes  have  been  added 
arbitrarily.  This  configuration  can  illustrate  the  problems  encountered  when  two  of  the  lowest- 
level  nodes  (i.e.,  29  and  3 or  1 and  29)  attempt  to  communicate  through  a number  of 
ascending  order  nodes. 


e . Loop  Node  Configuration 

The  Loop  Node  Configuration  (Figure  5.1-5)  consists  of  a number  of  cross- 
connected  loops.  This  particular  configuration  would  be  employed  in  a packet-switching  type 
system  where  the  links  looping  several  nodes  together  would  acts  as  a bus  to  carry  information 
around  the  loop.  In  this  node  configuration,  the  loops  are  cross-connected  such  that  the  loss 
of  any  one  link  will  not  isolate  any  node  and  the  loss  of  two  links  will  not  isolate  more  than 
one  node.  In  this  particular  configuration,  Nodes  8,  10,  18,  and  25  have  more  interconnecting 
links  terminating  at  these  nodes  than  the  other  nodes.  This  would  indicate  these  nodes  originated 
more  traffic  than  the  other  nodes. 

5.2  Other  Network  Configuration  Parameters 

Each  of  the  network  configurations  consists  of  thirty  nodes.  The  choice  of  30  node 
networks  was  made  as  a realistic  compromise  between  computational  capability  (in  the  simula- 
tion) and  the  achievement  of  relevant  results. 

The  network  connectivity  patterns  described  above  are  key  elements  in  specifying 
network  configurations  but  they  do  not  completely  specify  the  configuration.  In  order  to 
accomplish  this,  the  type  of  link,  link  bandwidth  and  link  distance  must  be  specified  for  each 
pair  of  connected  nodes.  The  types  of  links  that  will  be  included  in  the  network  configuration 
are  LOS  Microwave,  cable  and  wire  line,  tropospheric  and  satellite. 

Each  is  characterized  by  different  propagation  delay  variations.  The  link  distances 
will  vary  from  approximately  ^, 000  m iles  for  a satellite  link  to  perhaps  10  miles  for  the  wire  or 
microwave  link  between  a lower  level  node  and  the  node  to  which  it  is  slaved.  The  principal 
bit  rate  for  the  various  links  will  be  1 .544  Mb/s. 

For  each  of  the  connectivity  diagrams  shown,  appropriate  combinations  of  link 
media,  bit  rates,  and  distances  have  been  chosen.  These  are  described  in  the  section  on 
simulation.  Thus,  a single  connectivity  diagram  may  define,  by  changing  these  parameters, 
one,  two,  three  or  more  network  configurations.  Each  network  configuration  so  defined  will 
serve  as  a test  bed  in  the  computer  simulation  to  aid  in  the  evaluation  of  the  performance  of  the 
candidate  timing  techniques. 
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Figure  5.1-5.  Loop  Node  Configuration 
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OPERATING  AND  STRESS  SCENARIOS 


6.1  Introduction 

The  ultimate  objective  of  the  scenario  approach  is  to  provide  rigor  to  the  process 
whereby  a candidate  technique  or  combination  of  techniques  may  be  selected  on  the  basis  of 
performance,  cost  and  other  parameters  for  deployment  as  a prototype  timing  subsystem  with 
reasonable  assurance  that  the  proper  parameters  have  been  addressed  with  appropriate  emphasis. 
The  ground  rules  for  the  general  scenario  development  program  include: 

1 . Scenarios  will  postulate  an  initial  benign  environment  and  progress  through 
discrete  changes  in  environment  to  maximum  levels  of  anticipated  stress. 

2.  Scenarios  will  be  designed  to  address  a limited  number  of  parameters  at  any 
one  time,  holding  all  other  parameters  constant  - thereby  addressing  a 
limited  number  of  variables  at  any  one  time. 

3.  Scenarios  must  accommodate  a dynamic  environment  where  stresses  cause 
transition  (or  no  transition)  from  a prestress  steady  state  to  a poststress  state, 
which  may  be  stable  or  unstable. 

4.  Scenarios  must  pose  stresses  that  are  relevant  and  representative  and  approach 
the  real  world  environment. 

As  indicated  by  the  ground  rules  above,  it  is  appropriate  to  evaluate  the  relative 
performance  of  the  timing  schemes  inti ti ally  in  a peacetime  environment  and  progress  from  the 
most  minor  perturbations  on  simple  network  configurations  to  more  dramatic  perturbations  on  a 
more  complex  network.  In  order  to  address  a limited  number  of  parameters  at  any  one  time,  it 
is  further  proposed  to  place  geographical  constraints  on  the  scenarios,  with  geographically- 
oriented  groups  of  scenarios  being  exercised  prior  to  attempting  to  assess  interarea  operability 
of  the  NTSS.  The  geographical  constraint  leads  to  the  three  main  areas  of  the  CONUS,  Europe 
and  the  Pacific  region. 
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One  group  of  scenarios  then,  will  consider  operation  within  the  CONUS  in 
peacetime  in  what  is  considered  a situation  of  normal  stress.  Equipment  failure  and  natural 
phenomena  cause  node  outages,  weather  causes  microwave  fading,  lightning  is  an  ever- 
present source  of  power  outage  or  line  surges,  and  maintenance  problems,  aging  devices, 
etc.,  introduce  degraded  operation. 

6.2  Types  of  Perturbations 

Three  types  or  categories  of  perturbations  will  be  examined.  These  are  the  loss 
of  a node,  a change  in  clock  frequency  and  link  anomolies.  For  each  category  there  are  at 
least  two  and  perhaps  several  specific  perturbations  within  the  category.  With  respect  to  the 
loss  of  a node  either  a major  or  an  intermediate  node  may  be  considered.  For  changes  in 
clock  frequency,  an  implusive  change,  a step  change,  or  a change  in  drift  rate  may  be 
considered.  In  the  case  of  an  impulsive  change  it  may  take  place  at  a single  node  or  simul- 
taneously at  a large  number  of  nodes.  The  link  anomolies  category  would  include  fading 
on  a microwave  link,  path  delay  variations  and  jamming. 

Perturbations  that  can  impact  the  primary  and/or  secondary  timing  standard  and 
associated  circuitry  in  the  peacetime  "normal  operation"  mode  include  problems  of  routine 
operation  and  maintenance  and  the  inherent  stability  (drift,  aging)  of  the  devices  involved. 

In  addition,  the  effects  of  direct  nuclear  radiation  (X-rays,  neutrons,  and  gamma-rays) 
as  well  as  the  electromagnetic  effects  of  the  electromagnetic  pulse  (EMP)  must  be  considered. 
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Those  perturbations  that  impact  the  communications  node  itself  and  which  can 
cause  a collateral  effect  on  the  timing  device  or  NTSS  technique  include  lightning,  with 
associated  power  related  and  computer  operation  problems,  and  other  natural  phenomena 
(flood,  fire,  etc.)  leading  to  more  catastrophic  impacts.  Also,  prime  power  supply  maintenance 
can  impact  the  operation  of  the  node  as  well  as  poor  technical  control  orderwire  procedures. 

With  reference  to  perturbations  occurring  external  to  the  node,  the  most  dramatic 
impact  is  expected  to  result  from  links  interconnecting  the  node  under  evaluation  with  the 
remainder  of  the  system  and  at  nodes  directly  linked  to  the  node  in  question.  In  terms  of  the 
linking  problem,  loss  of  timing  information  can  result  from  natural  foding  phenomena  in  the 
peacetime  "normal  operation"  scenarios. 
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The  loss  of  a node,  In  the  majority  of  the  NTSS  techniques,  will  require  the 
exercising  of  predetermined  operations,  possibly  to  choose  an  alternate  route  to  derive  timing, 
call  upon  the  use  of  the  local  timing  source  for  a temporary  period  of  time,  etc.  The  loss 
of  an  adjoining  node  may  cause  either  a degraded  mode  of  operation  for  a period  of  time  or 
initial  acceptable  performance  which  degrades  with  time. 

6.3  Scenarios 

The  subsections  that  follow  list  the  scenarios  that  were  considered.  In  each  sub- 
section the  scenario  is  described  by  the  event  or  series  of  events  that  take  place;  the  resultant 
stress  that  is  placed  on  the  timing  subsystem  and  the  rationale  for  the  inclusion  of  the  particular 
scenario. 

Scenario  A 

This  scenario  is  node  oriented  and  applicable  to  peacetime,  CONUS,  operation. 
Assume  that  improper  maintenance  procedures  on  a cesium  beam  tube  primary  standard  at  a major 
node  result  in  the  failure  to  degauss  the  electromagnetic  shield. 

Stress 

The  aging  rate  of  the  cesium  standard  changes  over  a period  of  1 day  to  a figure 
of  5 x 10  ^ per  24  hours,  the  aging  rate  of  the  associated  quartz  oscillator. 

Rationale 

In  order  to  isolate  the  c-field  from  the  earth's  magnetic  field  and  external 
electromagnetic  environments  (radar,  communications,  transmitters,  etc.,)  it  is  necessary 
to  enclose  the  tube  of  the  standard  in  electromagnetic  shielding  material  and  degauss  the  shield 
at  periodic  intervals  to  guard  against  saturation.  Failure  to  do  so  will  impact  the  c-field  and 
cause  a change  in  frequency  of  the  primary  standard.  It  is  assumed  that  as  soon  as  the  threshold 
of  electromagnetic  isolation  is  reached,  i.e.,  saturation,  the  c-field  will  be  impacted  by  the 
surrounding  electromagnetic  environment  with  a resultant  change  in  basic  frequency.  As  the 
external  environment  changes,  the  frequency  of  the  atomic  standard  will  change  and  the  crystal 
oscillator  will  track  within  its  limits  of  control.  If  these  limits  are  exceeded,  the  assumption  is 
made  that  the  aging  rate  of  the  device  becomes  the  aging  rate  of  the  associated  quartz  oscillator. 
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Maintenance  problems  are  realistic  to  assume  and  some  will  surely  cause  a change 
in  the  frequency  of  the  primary  standard.  This  scenario  should  exercise  the  effectiveness  of  all 
of  the  timing  schemes  anticipated  for  possible  employment. 

Scenario  B 

This  scenario  is  node  oriented  and  applicable  to  peacetime,  CONUS  operation. 

Assume  that  a major  node  of  the  DCS  employs  a cesium  standard  in  which  com- 
ponent parameters  in  the  dc  supply  change  with  age. 

Stress 

The  inherent  stability  of  the  standard  changes  from  ±1x10  ^ for  the  life  of  the 
beam  tube  to  ±1  x 10  per  month. 

Rational 

The  output  of  the  primary  atomic  standard  is  derived  from  a highly  stable  5 MHz 
voltage  controlled  crystal  oscillator  whose  frequency  is  locked  to  the  invariant  resonance  of  the 
cesium  atom  undergoing  a change  in  atomic  state  in  the  beam  tube.  The  change  in  atomic 
state  is  highly  dependent  upon  the  application  of  a very  precisely  controlled  magnetic  field 
which,  in  turn,  is  set  up  by  a precise  dc  supply.  Any  stress  applied  to  the  power  supply  which 
degrades  its  performance  will  impact  the  precise  frequency  of  the  cesium  beam  tube,  thereby 
changing  the  error  signal  fed  back  to  the  voltage-controlled  crystal  oscillator  and  impacting 
the  output  frequency.  For  purposes  of  this  discussion,  we  will  assume  aging  of  components  in 
the  dc  supply  as  the  perturbing  source  which  results  in  an  aging  figure  more  appropriate  to  a 
rubidium  secondary  standard. 

Aging  of  components  can  be  expected  even  in  very  precise  and  well  controlled 
equipments  such  as  the  cesium  primary  standard.  Normally,  an  alarm  would  show  this  out  of 
specification  condition,  however,  poor  maintenance  procedures  could  subvert  this  safety 
feature. 
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Scenario  C 

This  scenario  is  system  oriented  and  is  applicable  primarily  to  wartime;  however, 
it  could  occur  in  peacetime  or  as  a precursor  to  war.  In  this  case,  there  would  be  no  wartime 
collateral  effects. 

Assume  that  a nuclear  device  is  detonated  exoatmospherically.  All  satellites 
within  direct  line-of-sight  of  the  burst  will  be  perturbed  by  the  effects  of  direct  radiation  and 
all  ground-based  electronics  will  feel  the  effects  of  the  classical  electromagnetic  pulse  (EMP). 

Stress 

All  unhardened  satellites  will  be  rendered  inoperative  and  will  not  return  to 
operation.  The  minimum  impact  to  ground-based  facilities  is  assumed  to  be  random  pulses 
of  voltage  and  current  on  signal  lines,  power  lines,  computers  and  peripherals  such  that  every 
node  is  rendered  inoperative  for  a period  of  a few  hundred  milliseconds.  All  buffers  are 
scrambled  and  all  crypto  synchronization  is  lost  simultaneously  over  the  entire  system.  No 
permanent  damage  results  to  nodal  equipment. 

Rationale 

This  scenario  can  exist  either  as  a result  of  a nontreaty  member  testing  a nuclear 
device  exoatmospherically  in  peacetime  or  the  U.S.  or  an  adversary  detonating  such  a device 
"for  effect"  as  a precursor  to  warfare.  The  low  level  of  stress  assumed  for  the  ground  based 
system  is  a minimum  stress  scenario  for  high  altitude  EMP  effects.  Satellites  will  survive  or 
be  lost  depending  upon  levels  of  hardness  and  specific  satellite  links  will  be  designated  as 
lost  in  the  scenario.  Reconstitution  time  of  the  ground  based  facility  timing  subsystem  and  the 
ease  of  resynchronization  will  be  evaluated. 

Scenario  D 

This  scenario  is  node  oriented  and  is  applicable  to  wartime. 

Assume  that  a backbone  node  is  attacked  by  a team  of  saboteurs.  The  communica- 
tion facility  is  completely  and  permanently  disabled.  Alternate  routing  capabilities  exist  for 
information  transfer  to  all  other  major  nodes  of  the  system. 
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Stress 

The  loss  of  the  node  will  require  rerouting  of  traffic,  traffic  delays  and  may 
require  alternate  paths  for  the  distribution  of  timing. 

Rationale 

This  is  a minimum  stress  scenario  when  taken  in  context  with  an  entire  commu- 
nication system.  Many  nodes  could  be  lost  nearly  simultaneously  as  a result  of  sabotage  and 
direct  attack. 

Scenario  E 

This  scenario  is  system  oriented  and  is  applicable  to  wartime  - OCONUS. 

Assume  a determined  jamming  attack  by  the  enemy  which  initially  disables  one 
backbone  node,  followed  by  a multiplicity  of  nodes  as  time  progresses  until  25  percent  of  the 
nodes  of  the  system  are  inoperative.  At  a predetermined  time  all  jamming  ceases  and  all 
nodes  return  to  normal  operational  capability  simultaneously. 

Stress 

There  is  gradual  increase  in  stress  on  a system  with  time  until  the  system  reaches 
the  maximum  degree  of  inoperability.  The  system  is  agcin  stressed  as  relinking  occurs  at  the 
end  of  the  jamming  attack. 

Rationale 

This  is  a routine  wartime  stress  on  any  communication  system  located  within 
range  of  ground  based  enemy  jammers.  The  perturbations  to  the  nodal  timing  at  nodes  not 
directly  affected  by  the  jamming  will  vary  between  timing  subsystems.  The  magnitude,  dura- 
x-  ,nd  recovery  from  this  perturbation  is  of  concern. 

Scenario  F 

'•  is  scenario  is  link  oriented  and  is  applicable  to  peacetime. 

me  that  as  a result  of  atmospheric  phenomena,  a microwave  communications 
n<  -bone  nodes  of  the  DCS  experiences  typical  signal  fading  such  that  bit 
r.  »ion  of  time  until  such  time  as  the  errors  become  excessive. 
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Stress 

The  random  variation  and  gradual  increase  in  bit  error  rate  will  result  in  errors 
in  timing  subsystem  control  information  and  will  ultimately  result  in  loss  of  the  link. 

Rationale 

This  problem  must  be  addressed  routinely  by  any  communications  system  using 
microwave  links;  however,  its  effect  on  the  system  would  be  much  less  pronounced  than  the 
loss  of  a physical  communications  node. 

Scenario  G 

This  scenario  is  link  oriented  and  applicable  specifically  to  time  reference 
distribution.  Assume  that  a major  node  of  the  DCS  transfers  time  through  a satellite  link  and 
the  satellite  fails.  The  node  then  shifts  to  a backup  mode  of  access  to  UTC,  i.e.,  LORAN-C. 

Stress 

The  accuracy  with  which  absolute  time  can  be  transferred  from  this  node  through- 
out the  system  changes  from  a figure  of  100  nanoseconds  for  the  satellite  path  to  50  microseconds 
for  LORAN-C. 

Rationale 

This  scenario  is  important  to  an  assessment  of  the  Time  Reference  Distribution 
Technique  for  peacetime  evaluation.  The  tolerance  figures  would  change  since  the  access 
route  to  UTC  would  change  from  a satellite  link  t<">  OMEGA,  a future  OCONUS  LORAN-C, 
etc.  It  is  realistic  since  many  stresses  will  cause  changes  in  the  choice  of  paths  through 
intervening  nodes  and  links  to  a source  of  UTC.  Each  of  the  paths  will  offer  a maximum 
optimum  figure  for  absolute  synchronization. 

We  anticipate  showing  that  the  inclusion  of  Precise  Time  Availability  causes  no 
added  vulnerability  to  the  system;  on  the  contrary,  it  adds  an  important  capability  to  the  DCS, 
both  CONUS  and  OCONUS.  This  scenario  will  help  determine  the  applicability  of  an 
external  time  reference  as  a backup  to  a time  reference  distribution  approach. 
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Scenario  H 


This  scenario  is  node  oriented  and  applicable  to  peacetime,  CONUS  operation. 

Assume  that  within  a specific  level  inanodal  hierarchy  VCXO's  are  utilized 
as  the  clocks  at  each  of  the  nodes.  The  short  term  stability  of  these  clocks  is  1 x 10 
Due  to  a local  electrical  power  transient  the  short  term  stability  of  one  clock  degrades  to 
1 x 10"6. 

Rationale 

Such  changes  in  crystal  oscillators  are  possible  as  a result  of  both  electrical 
and  mechanical  transients.  How  this  perturbation  at  this  single  node  is  manifested  in  the 
clocks  at  other  nodes  will  vary  according  to  timing  technique.  It  is  these  variations  that 
will  be  examined. 
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SIMULATION 


7.1  Goals 

The  purpose  of  the  simulation  is  to  provide  a means  for  studying  the  reaction  of  a 
complex  interconnected  network  to  various  disturbances  and  stresses.  It  is  mainly  due  to  the 
numbers  of  nodes  and  links  that  must  be  used  in  a representative  model  that  a computer  simula- 
tion is  deemed  necessary.  The  two  or  three  node  case  can  be  analyzed  without  such  aids. 

Normal  disturbances  include  clock  jitter,  clock  drift,  initial  frequency  and  phase  offsets  and 
normal  link  delay  variations.  Abnormal  stresses  include  clock  failures,  node  failures,  link 
failures  and  abnormal  link  delay  variations.  The  simulator  can  also  be  used  as  a design  tool 
by  obtaining  fine  grained  information  for  selection  of  phase-lock  loop  parameters,  buffer  sizes 
and  clock  stabilities.  Further,  the  simulator  can  be  used  for  comparing  the  performances  of 
various  timing  techniques.  The  major  parameters  used  in  such  comparisons  include  the 
fol  lowing: 

1 . The  speed  with  which  the  network  disposes  of  a transient  condition 

2.  The  degree  of  closeness  that  the  nodal  clock  frequencies  are  held  together 

3.  The  amount  of  phase  error  that  can  be  found  at  the  various  nodes 

During  the  period  of  recovery  from  a disturbance  occurring  somewhere  in  the  network,  the 
timing  subsystem  is  more  vulnerable  to  any  other  disturbances  that  might  occur.  During  transient 
conditions,  error  detection  equipment  has  a degraded  frame  of  reference  and  reconfiguration 
apparatus  is  more  susceptible  to  error  in  the  worst-case  and,  at  best,  is  subject  to  a delayed 
response.  Thus,  shorter  transient  time  is  a desirable  feature.  The  ideal  condition  is  that  all 
clocks  in  the  network  remain  exactly  together.  By  providing  receiving  buffers  on  each  link, 
some  tolerance  to  temporary  differences  in  clock  frequencies  may  be  obtained.  However,  any 
two  communicating  nodes  must  have  an  average  difference  frequency  of  zero  if  finite  size 
buffers  are  used  and  loss  of  data  is  avoided.  Thus,  clocks  held  closely  together  is  a desirable 
attribute.  Small  phase  error  at  any  instant  of  time  is  desirable  because  phase  error  represents 
a difference  in  clock  time  readings  and  is  stored  potential  which  may  be  converted  to  a 
kinetic  change  in  clock  frequencies  as  a result  of  some  future  disturbance. 
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7.2  Models 


I The  simulator  is  designed  to  model  master-slave,  mutual  sync,  independent  clocks, 

and  time  reference  distribution  techniques.  Each  model  consists  of  a set  of  nodes  connected 
together  by  a set  of  communications  links.  The  major  differences  between  the  techniques  are 
the  manners  in  which  reference  information  is  obtained  at  each  node  from  other  nodes  in  the 
network.  Secondary  differences  are  concerned  with  the  manner  in  which  this  information  is 
filtered  and  compared  with  the  local  clock  on  an  instantaneous  basis.  The  characteristics  of  the 
nodal  clocks  and  communications  links  are  the  same  no  matter  which  timing  technique  is  utilized. 
Therefore,  these  will  be  discussed  separately  from  the  individual  techniques. 

7.2.1  Timing  Techniques 


7. 2. 1.1  Independent  Clocks 

The  independent  clocks  technique  uses  no  reference  information,  hence,  the  name. 
Since  no  two  real  clocks  can  remain  together  for  more  than  a short  period,  occasional  buffer 
overflow  or  underflow  is  assured  with  this  technique.  Thus,  the  major  thing  of  interest  under 
this  scheme  is  the  rate  at  which  buffers  overflow.  This  information  can  then  be  used  to  set  up  a 
convenient  periodic  interval  within  which  data  buffers  may  be  reset  at  all  nodes  of  the  network. 
In  the  simple  case  in  which  each  clock  in  the  network  is  assumed  to  be  a constant  frequency 
offset  from  any  other  clock  in  the  network,  the  amount  of  time  to  overflow  a buffer  of  a given 
size  between  any  two  communicating  nodes  can  be  readily  determined  without  the  aid  of  a 
simulator.  Actually,  this  might  approximate  the  typical  situation  for  this  technique  since  high 
stability  clocks  would  probably  be  used  with  this  scheme,  and  hence  would  remain  essentially 
constant  over  a period  of  time  required  to  fill  up  a data  buffer.  Let  f^  and  fj  be  the  frequencies 
of  two  clocks  whose  nodes  are  assumed  to  be  communicating  and  assume  f^  is  the  reference. 
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where  N is  the  number  of  cycles  of  f. . Letting  fj  ~ 0 + 8)  ^ we  obtain 

(1  +8)f, 


T -f  'Tnrrr^  * * ‘ ( ' * 1 ) * * 


But  now  for  I S | < < 1 this  becomes  approximately 
N _ <f> 


T = 


f. 


(7.2. 1.1-3) 


(7. 2. 1.1-4) 


7. 2. 1.2  Master-Slave 
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7. 2. 1.2.1  Dissemination  of  Reference  Information 

The  master-slave  technique  has,  superimposed  on  the  communications  network,  a 
subset  of  links  in  the  form  of  a tree  structure  over  which  reference  information  for  timing 
purposes  is  disseminated.  In  the  model  being  described,  this  reference  timing  information  is 
assumed  to  be  obtained  from  the  bit  transitions  of  the  incoming  data  stream.  However,  this  is 
not  a necessary  restriction  on  the  timing  technique  since  one  data  channel  could  be  reserved 
to  carry  the  timing  information  in  some  other  form.  Under  this  scheme,  the  amount  of  variation 
in  the  reference  information  received  at  each  node  from  that  of  the  master  clock  should  increase 
as  the  tree  level  number  increases  due  to  intervening  link  delay  variations.  Some  attenuation 
or  amplification,  depending  on  nodal  filter  characteristics,  is  applied  to  the  variations  in  the 
reference  information  at  each  node  as  it  progresses  to  succeeding  levels  in  the  timing  tree. 

Thus,  the  aggregate  variation  seen  at  a particular  node  is  that  due  to  the  immediately  preceding 
link,  plus  that  which  leaked  through  the  preceding  node's  filter  plus  high  frequency  variations 
from  the  previous  node's  clock. 
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7. 2. 1.2. 2 Organization/Reorganization 

Network  organization  refers  to  the  structure  of  the  tree  over  which  the  reference 
timing  information  is  disseminated.  For  a given  network,  there  may  be  a very  large  number  of 
different  trees  which  can  be  formed.  The  goal  of  network  organization  is  to  provide  the  best 
reference  information  to  each  node.  Thus,  the  parameters  of  the  organization  scheme  are  link 
quality  and  nodal  clock  quality.  Fortunately,  the  operations  necessary  for  organization  can  be 
distributed  over  all  the  nodes  of  the  network.  This  is  true  since  each  node  and  link  can  be 
assigned  a normal  quality  factor  and  monitors  can  be  provided  to  detect  deviations  from  the 
norm.  A given  node  may  then  either  reference  itself  or  obtain  its  reference  information  from 
one  of  its  immediate  neighbors  via  that  neighbor's  communicating  link.  The  choice  is  for 
highest  clock  quality,  link  quality  product.  Reorganization  refers  to  implementing  a timing 
tree  structure  change  after  some  change  in  status  of  a node  or  link  of  the  network.  In  order  to 
give  the  simulator  user  complete  control  of  the  organization/reorganization  process  and  to 
simplify  the  simulation  model,  the  tree  structure  is  specified  by  the  user  via  the  input  data 
stream.  This  is  not  a particularly  burdensome  requirement  on  the  user  since  he  must  already  specify 
the  original  structure  and  condition  of  all  links  and  nodes  and  any  changes  thereto  that  are  to 
take  place  during  each  simulation  run  via  the  input  data  stream. 
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7. 2. 1.2. 3 Processing  the  Timing  Information 

A block  diagram  of  the  master-slave  node  is  shown  in  Figure  3.1.1. 

7. 2. 1.2. 3.1  Loop  Filters 

For  slave  nodes,  the  model  provides  for  either  Type  1 or  Type  2 filters.  The 
Type  1 filter  is  a single  pole  filter  with  transfer  function  of  the  form  + . The  Type  2 

filter  is  an  integral  plus  proportional  type  with  transfer  function  of  the  form  — — — . Use  of 
either  of  these  filters  in  a phase-lock  loop  arrangement  results  in  a second  order  loop.  The  major 
difference  is  that  the  Type  2 filter  is  able  to  track  a constant  offset  between  the  reference  signal 
frequency  and  the  natural  value  of  the  local  clock  frequency  with  a zero  average  phase  error, 
whereas  the  Type  1 filter  requires  an  average  phase  error  that  is  proportional  to  the  amount  of 
frequency  offset. 
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7. 2. 1.2. 3. 2 Phase  Detector 

The  model  uses  an  extended  range  linear  phase  detector.  Real  implementations  of 
this  might  utilize  the  receive  data  buffer  or  an  up/down  counter.  Some  performance  comparisons 
are  made  between  a 1 cycle  nonlinear  sinusoidal  phase  detector  and  the  extended  range  linear 
phase  detector  in  Appendix  A. 

7. 2. 1.2. 3. 3 Acquisition  Mode  Switching 

In  order  to  quickly  acquire  frequency  lock  from  a large  frequency  offset  as  might 

be  experienced  when  switching  from  one  reference  source  to  another  and  also  provide  some 

attenuation  of  daily  path  delay  variations,  two  sets  of  loop  parameters  are  provided  for  use  with 

the  Type  2 filter,  one  for  tracking  and  one  for  fast  acquisition.  The  method  for  switching  from 

tracking  parameters  to  acquisition  and  vice  versa  is  as  follows:  each  node  is  normally  in  its 

tracking  mode  until  its  phase  error  exceeds  a threshold  value  <j>  ^ whereupon  it  automatically 

switches  to  its  acquisition  parameters.  It  then  remains  in  the  acquisition  mode  for  a period  of 

time  T that  is  equal  to  several  times  the  acquisition  phase-lock  loop's  time  constant.  It  then 
acq 

switches  bock  to  the  tracking  parameters.  When  the  switch  is  made  from  tracking  to  acquisition, 
a step  change  in  frequency  of  magnitude  2 f in  the  direction  opposite  to  that  which 

originally  produced  4>  ^ is  experienced.  Normalized  plots  of  the  output  frequency  and  phase 
error  of  the  second  order  phase-lock  loop  using  an  extended  range  linear  phase  detector  and  the 
Type  2 filter  for  initial  phase  error  and  frequency  offset  are  shown  in  Appendix  A.  Also,  the 
scheme  does  not  ensure  zero  phase  error  and  zero  frequency  offset  before  switching  from 
acquisition  back  to  tracking  parameters.  However,  this  will  be  approximated  to  a sufficient 
degree  provided  no  large  disturbances  to  the  reference  information  occurs  during  the  acquisition 
interval.  The  major  advantage  of  this  mode  switching  scheme  is  its  simplicity  and  hence,  the 
ease  with  which  it  may  be  implemented. 


7. 2. 1.3 


Mutual  Sync 


7. 2. 1.3.1  Dissemination  of  Reference  Information 

The  mutual  sync  technique  was  originally  proposed  to  avoid  the  problems  of  network 
reorganization.  The  method  used  at  each  node  is  to  obtain  some  weighted  average  of  the 
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reference  information  from  all  of  its  immediate  neighboring  nodes.  Thus,  no  reorganization  is 
required  except  for  removing  any  failed  references  from  the  composite  error  signal. 

7. 2. 1.3. 2 Processing  the  Reference  Information 

A block  diagram  of  a mutual  sync  node  is  shown  in  Figure  3.2.1. 

7. 2. 1.3. 2.1  Phase  Detector 

The  mutual  sync  node  is  quite  similar  to  that  of  a master-slave  node  except  for  the 

method  of  obtaining  the  phase  error  voltage.  In  its  simplest  form,  a separate  phase  detector  is 

used  for  each  of  the  incoming  reference  signals.  The  individual  signals  are  then  weighted  in 

some  fashion  and  the  weighted  signals  are  summed  to  produce  the  composite  error  signal.  The 

1 N 

simplest  weighting  scheme  is  to  weight  all  signals  equally,  i.e.,  4>  - - — L <Pej  . 

e IN  ; = l 

The  model  utilizes  extended  range  linear  phase  detectors  for  obtaining  the  phase  error  estimates. 


7. 2. 1.3. 2. 2 Loop  Filter 

Since  all  the  nodes  of  the  network  are  connected  together  in  a feedback  arronge- 
ment,  one  must  be  careful  in  selecting  the  loop  filters  in  order  to  avoid  problems  of  instability. 
Basically,  one  must  ensure  that  the  closed  loop  gains  of  the  individual  nodes  are  less  than  1 rnd 
that  the  gain-delay  products  are  less  than  1 . The  stability  problem  is  discussed  more  fully  in 
Section  3.0.  The  Type  2 filter  does  not  meet  the  stability  criteria  but  the  Type  1 does  for  loop 
damping  ratios  greater  than  Thus,  the  model  utilizes  Type  1 filters.  Since  the  Type  1 

filter  requires  a phase  error  proportional  to  the  offset  between  the  reference  frequency  and  the 
local  clock  frequency,  and  individual  error  signal  may  be  weighted  by  some  factor  considerably 
less  than  1,  phase  error  much  larger  than  required  for  master-slave  may  exist  in  the  mutual  sync 
system.  Should  a reference  source  fail  which  has  a large  phase  error,  simple  removal  of  this 
source  from  the  composite  error  signal  without  careful  reweighting  of  the  remaining  sources  can 
result  in  a large  change  in  local  clock  frequency.  It  should  also  be  noted  that  because  the 
entire  network  is  interconnected  in  a feedback  arrangement,  it  requires  considerably  longer  for 
steady-state  conditions  to  be  reached  following  network  disturbances  than  with  the  master-slave 
technique  if  the  same  filter  time  constants  are  used. 
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7.2.1  .4 


Time  Reference  Distribution  (TRD) 


7.2.1  .4.1  Dissemination  of  Reference  Information 

The  TRD  technique  utilizes  the  same  kind  of  structure  for  disseminating  reference 
information  as  is  used  for  the  master-slave  technique.  However,  it  is  a double  ended  scheme, 
i.e.,  there  is  a two  way  exchange  of  information  between  adjacent  nodes  in  a chain  such  that 
the  effects  of  transmission  delay  variations  are  removed.  In  this  manner  the  nodal  clocks  may  be 
held  as  closely  togethei  as  two  way  path  delay  asymmetries,  and  measurement  errors  allow.  The 
model  assumes  that  a special  overhead  channel  is  utilized  for  carrying  the  reference  information. 

A further  difference  between  the  TRD  and  master-slave  techniques  is  that  reference  information 
concerning  an  estimated  error  between  the  ultimate  reference  source  and  each  node  in  the  chain 
is  added  in  and  passed  on  to  each  succeeding  node  in  the  chain.  This  information  is  not  filtered 
before  being  utilized. 

7.2.1  .4.2  Organization/Reorganization 

The  model  utilizes  an  adaptive  self-organizing  scheme  based  on  a system  of  clock 
quality  factors  and  link  demerits.  The  reference  information  is  passed  down  each  chain  at  discrete 
time  intervals.  Thus,  the  information  becomes  progressively  older  as  it  ascends  to  higher  levels 
of  the  distribution  tree.  This  is  a source  of  a problem  for  the  reorganization  and  clock  correction 
schemes  called  the  obsolete  data  problem.  In  order  to  avoid  errors  due  to  this  phenomenon,  the 
reorganization  and  correction  schemes  must  examine  enough  information  samples  to  be  sure  of  its 
validity  before  utilizing  it.  The  effects  of  this  problem  on  clock  correction  can  be  mitigated 
somewhat  if  correction  extends  over  many  more  samples  than  there  are  intervening  nodes  between 
the  node  in  question  and  its  ultimate  reference. 


7. 2. 1.4. 3 Processing  the  Reference  Information 


A block  diagram  of  the  TRD  node  is  shown  in  Figure  3.3.1  . 
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7. 2. 1.4. 3.1  Phase  Detector 

The  TRD  technique  utilizes  a discrete  form  of  double  ended  linear  phase  detector. 
A special  overhead  channel  is  assumed  to  be  available  for  periodically  exchanging  a set  of 
reference  information  messages  between  adjacent  nodes  in  each  path  of  the  tree.  The  exchange 
sequence  is  described  as  follows:  let  A and  B be  nodes  in  the  same  timing  chain  and  let  A be 
closer  to  the  ultimate  reference  source  than  B.  Node  B sends  a message  to  Node  A containing 
the  time  T^  on  Node  B's  clock.  After  a delay  of  D^,  the  transmission  delay  from  B to  A, 
Node  A receives  this  message.  Upon  receiving  this  message  Node  A notes  his  own  time  T^ 
and  forms  the  quantity  «A  = T^  - (T^  - D^).  Node  A then  transmits  a message  to  Node  B 
containing  the  reading  on  Node  A's  clock  Ta2  an<^  f^e  quantity  K^.  After  a delay  D^,  the 
transmission  delay  from  A to  B,  Node  B receives  this  message,  notes  his  own  clock  reading  Tg2 
and  forms  the  quantity  KR  = T^  - (TA2  - D^).  Node  B then  calculates  the  quantity 

KA  ~ KB  _ TA1  ~ (TB1  " °BA}  ~ [TB2  " (TA2  ~ DAB}] 


(TA1  “ TB1 ] + (TA2  ' TB2)  + °BA  " °AB 


Now,  since  the  clocks  are  assumed  to  drift  very  little  during  the  exchange  interval, 
(T  - T^  ) and  (TA2  - T^2)  are  approximately  equal  to  each  other . Thus 


ka~kb 


2 ta"tb  + 


dba  " dab 


Therefore 


TB  ” 


ka-kb 


D 


AB 


D 


BA 


If  the  term  (D  - D A)/2  which  represents  the  asymmetry  in  the  two  directions  of  the  communi- 
AB  BA 

cations  link  is  ignored,  then  the  error  of  this  process  is  that  due  to  this  asymmetry  plus  the  error 
in  recognizing  the  epochs  at  which  messages  are  received  plus  any  clock  drifts  during  the  exchange 
interval.  Node  B may  use  the  estimated  value  (TA  - T^)  in  order  to  correct  his  own  clock  so  that 
it  corresponds  to  that  of  Node  A.  The  model  takes  into  account  all  three  of  the  above  error 
sources  by  allowing  for  specification  of  a single  asymmetry  term  for  each  communicating  link. 
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In  addition  to  the  error  between  Node  B's  clock  and  that  of  Node  A,  obtained 
from  the  above  exchange  of  messages,  each  node  in  each  chain,  beginning  from  those  closest 
to  the  ultimate  network  master,  maintains  an  instantaneous  estimate  of  error  between  its  clock 
and  the  ultimate  reference  which  it  passes  on  to  all  nodes  at  higher  branches  of  the  tree.  Thus, 
Node  A also  passes  on  to  Node  B an  instantaneous  estimate  of  Node  A's  error  with  respect  to  the 
ultimate  master  which  Node  B adds  to  the  error  just  calculated  between  B and  A.  Node  B is  then 
able  to  track  the  ultimate  reference  without  having  to  follow  perturbations  occurring  at  interven- 
ing nodes.  The  model  takes  this  error  with  respect  to  the  ultimate  reference  into  consideration. 

7. 2. 1.4. 3. 2 Clock  Correction 

There  are  a number  of  techniques  for  processing  the  phase  error  samples  in  order  to 
obtain  a correction  signal  to  be  applied  to  the  local  clock.  The  type  of  processing  of  the  samples 
depends  somewhat  on  the  characteristics  of  the  local  clock.  For  example,  cesium  clocks  may 
require  a highly  filtered  correction  signal  in  order  to  take  advantage  of  their  excellent  long  term 
frequency  stability.  Two  cesium  clocks  operating  with  a constant  frequency  offset  from  each 
other  of  2 parts  in  10^  accumulate  phase  error  at  the  rate 

0 = 2 • lO'11  • t 

e 

Thus,  two  such  clocks  will  require  50,000  seconds  to  accumulate  a phase  error  of  1 ps.  In  light 
of  the  required  response  time  it  is  conceivable  that  the  phase  error  of  such  clocks  could  be  main- 
tained within  acceptable  bounds  by  a human  operator.  However,  it  is  believed  that  an  auto- 
mated scheme  is  preferred.  On  the  other  hand,  high  quality  quartz  crystal  clocks  have  a fre- 
quency drift  of  approximately  1 part  in  10^  per  day.  If  a node  having  a quartz  clock  with  this 
drift  rate  were  communicating  with  a node  using  a cesium  clock  and  they  were  started  at  time 
T = 0 with  zero  frequency  and  phase  error  then  phase  difference  would  accumulate  as 

1 5 2 

0 = 1.157  • 10  ° • t . 

e 

One  microsecond  of  phase  difference  would  accumulate  in  29, 400  seconds . Thus,  one  difference 
in  the  requirements  for  correction  signals  for  the  cesium  and  quartz  crystal  clocks  is  that  the 
quartz  clock  will  require  both  phase  and  frequency  correction  while  the  cesium  clock  may  require 
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only  phase  correction.  It  is  believed,  however,  that  frequency  correction  can  improve  the 
performance  of  the  cesium  clock. 

The  method  of  application  of  the  correction  signal  to  the  local  clock  depends  on 
whether  a reference  is  presently  available  and  whether  or  not  one  was  available  in  the  immediate 
past.  The  conditions  of  interest  in  regards  to  the  reference  are  as  follows: 

1 . Have  one  now  and  have  had  same  for  some  time. 

2.  Do  not  have  one  now  but  have  had  one  recently. 

3.  Have  one  now  but  have  not  had  same  for  very  long. 

The  correction  procedure  for  1.  was  covered  above.  In  case  2.  it  is  desirable  that  the  nodal 
clock  should  have  the  ability  to  coast  through  certain  reference  outages  without  accumulating 
appreciable  phase  error.  For  the  cesium  clock  this  means  that  a long  term  average.  A,  of  the 
previous  samples  should  be  applied.  This  same  technique  does  not  work  as  well  with  the  auartz 
crystal  clock  because  of  its  drift.  What  is  needed  with  the  quartz  crystal  clock  is  a long  term 
average  of  the  slope,  S,  of  the  previous  samples  as  well  as  knowledge  of  the  value  B of  the 
correction  signal  at  some  time  T.  During  the  coasting  period  the  correction  signal  B + 5 • (t  - T) 
can  then  be  applied. 

In  case  3.  the  ability  to  quickly  acquire  from  fairly  large  offsets  is  desired.  This 
mode  is  used  when  switching  references  or  when  bringing  a node  into  the  network  when  the  node 
has  been  off  for  some  reason. 

Except  for  case  2.  with  quartz  crystal  clocks,  a phase-lock  loop  employing  an 
integral  plus  proportional  filter  can  be  configured,  by  suitable  choice  of  filter  parameters,  to 
satisfy  most  of  the  above  requirements.  The  following  discussion  is  intended  to  show  how  one 
might  be  led  through  a succession  of  refinements  to  such  a scheme. 

205 

In  order  to  make  phase  error  corrections,  particularly  for  cesium  clocks,  Stover 
initially  suggested  measuring  phase  error  at  each  node  with  reference  to  the  master  clock  and 
then  providing  a discrete  step  correction  after  the  error  had  reached  some  threshold  level.  The 
correction  could  consist  of  either  a step  change  in  the  local  clock  frequency  for  some  period  of 
time  or  a ramp  phase  change  at  the  output  of  the  clock  for  the  same  period  of  time.  The  latter 
method's  advantage  is  that  the  clock  is  not  disturbed.  This  scheme  is  similar  in  operation  to  a 
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discrete  type  of  first  order  phase-lock  loop  since  it  will  result  in  a steady  state  average  phase 
error  different  from  zero  when  a constant  offset  exists  between  the  reference  signal  and  the 
natural  frequency  of  the  local  clock.  A block  diagram  of  this  system  is  shown  in  Figure 
7.2. 1 .4. 3.2-1 . It  should  be  noted  that  this  scheme  has  no  provision  for  systematic  frequency 
error. 


Since  systematic  frequency  error  is  one  of  the  major  contributors  to  the  need  for 

205 

phase  error  corrections,  it  is  believed  that,  as  suggested  by  Stover  particularly  for  quartz 
crystal  clocks,  the  above  simple  scheme  may  be  improved  by  making  provisions  for  systematic 
frequency  error  correction.  This  may  be  accomplished  by  taking  a discrete  derivative  of  the 
phase  error  samples  and  applying  this  derivative  signal  permanently  each  time  a phase  correction 
is  made.  Figure  7.2.1 .4. 3. 2-2  shows  a block  diagram  of  this  technique. 

Quantization  errors  in  the  scheme  of  Figure  7.2. 1 .4. 3. 2-2  can  be  reduced  by 
making  more  frequent  phase  error  measurements  and  lowering  the  phase  error  threshold  above 
which  corrections  will  be  made.  There  appears  to  be  no  theoretical  limit  in  the  smallness  of 
step  size  of  the  corrections.  Thus,  this  scheme  can  be  made  to  appear  to  the  user  of  the  local 
clock  as  a continuous  process.  However,  the  same  result  can  be  accomplished  at  reduced 
sampling  and  processing  cost  by  removing  the  phase  threshold  detector  and  the  frequency  cor- 
fection  threshold  enable  gate  and  leaving  phase  error  corrections  applied  permanently,  i.e., 
pass  the  phase  error  corrections  through  an  accumulator  (discrete  type  of  integrator).  What  was 
a frequency  correction  in  Figure  7.2. 1 .4. 3. 2-2  now  becomes  a proportional  term  and  the 
accumulated  phase  error  becomes  an  integral  term  with  these  modifications,  as  shown  in  Figure 
7.2.1 .4. 3. 2-3.  Scrutiny  of  Figure  7.2. 1 .4. 3. 2-3  reveals  that  it  has  the  form  of  a discrete 
phase-lock  loop  employing  an  integral  plus  proportional  loop  filter.  A more  conventional 
drawing  of  this  scheme  is  shown  in  Figure  7.2. 1 .4. 3. 2-4.  Some  advantages  of  this  scheme 
over  those  from  which  it  was  evolved  are  the  cost  of  its  implementation  and  its  recognizable 
potential  for  performance  analysis  by  well  known  techniques.  Three  parameters  completely 
describe  the  performance  of  this  scheme.  They  are: 


a.  Sampling  rate  of  the  phase  error  estimator 

b.  Scale  factor  on  the  integral  term  (as  in  Figure  7.2. 1 .4. 3. 2-4) 

c.  Scale  factor  on  the  sum  of  the  integral  and  proportional  terms  (K  in  Figure 
7. 2. 1.4. 3. 2-4) 
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Figure  7.2. 1 .4. 3. 2-1 . A Simple  Scheme  for  TRD  Clock  Correction 
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Figure  7.2. 1 .4. 3. 2-3.  TRD  Clock  Corrector  With  Integral  Plus  Proportional 
Filtering  of  Phase  Error  Estimates 


Figure  7.2. 1 .4. 3. 2-4.  Phase-Lock  Loop  Version  of  Figure  7.2.  1 .4. 3. 2-3 
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Parameters  a and  K can  be  translated  into  familiar  second-order  closed  loop  damping  ( £ ) and 
undamped  natural  frequency  (cun)  parameters  of  feedback  control  theory.  The  sampling  rate  of 
the  phase  error  estimator  must  be  consistent  with  the  loop  filter  characteristics,  i.e.,  the  values 
of  a and  K.  A discussion  of  the  required  sampling  rates  was  given  in  Paragraph  3.3.2. 1 of 
this  report.  If  quartz  crystal  clocks  are  used  in  the  TRD  network  then  a set  of  acquisition  param- 
eters are  required  in  addition  to  tracking  parameters  due  to  the  inherent  instability  of  these 
devices.  Cesium  clocks  need  only  be  provided  with  a set  of  tracking  parameters  because  their 
intrinsic  stability  is  quite  high.  The  wider  bandwidth  in  the  quartz  clock  acquisition  mode  sets 
a lower  bound  on  the  sampling  rate  of  the  phase  error  estimator.  The  drift  rate  of  the  crystal 
clock  sets  a lower  bound  on  the  loop  bandwidth  in  the  tracking  mode.  Normally,  this  bandwidth 
is  made  as  narrow  as  possible  in  order  to  maintain  a good  estimate  of  average  reference  frequency 
so  that  the  local  clock  never  tracks  very  far  from  this  long-term  average.  Thus,  it  is  better  able 
to  coast  through  a reference  outage  with  lower  probability  of  bit  slips.  The  bandwidth  damping 
ratio  product  should  also  be  kept  small  so  that  initial  phase  offsets  experienced  when  entering  the 
network  or  switching  references  do  not  cause  large  peak  frequency  transients  of  the  local  clock. 
Large  peak  frequency  transients  are  particularly  to  be  avoided  in  case  the  local  clock  is  a cesium 
standard.  On  the  other  hand,  extremely  narrow  bandwidth  will  cause  the  buildup  of  lorge  peak 
phase  errors  as  a result  of  step  changes  in  reference  frequency  such  as  might  be  experienced  when 
switching  from  one  reference  to  the  next.  Thus,  it  appears  that  the  quartz  and  cesium  clocks 
require  different  sets  of  tracking  loop  parameters.  The  quartz  crystal  clock  also  requires  a set 
of  acquisition  parameters  in  order  to  rapidly  acquire  from  large  offsets.  Sets  of  parameters 
which  take  into  account  the  factors  discussed  are  shown  in  Table  3.3.2. 1. 

7.2. 1.4. 3. 3 Acquisition  Mode  Switching 

The  TRD  model  utilizes  the  same  scheme  for  switching  from  tracking  parameters 
to  fast  acquisition  as  described  for  the  moster/slave  technique  in  Paragraph  7.2.1 .2.3.3. 


7-15 


7.2.2 


Normal  Link  Variations 


The  discussion  in  this  section  is  aimed  at  obtaining  some  reasonable  descriptions 
of  path  length  variations.  However,  very  rough  assumptions  are  made  in  order  to  keep  the 
models  as  simple  as  possible.  The  fact  that  values  chosen  may  not  meet  some  particular  situation 
does  not  necessarily  invalidate  the  model  since  results  can  be  scaled  in  accordance  with  the 
ratios  of  the  correct  values  or  parameters  to  those  assumed  in  the  model.  The  discussions  are 
based  primarily  on  referenced  intormation . ' 


7.2.2. 1 Cable 

Assume  buried  coaxial  cable  with  polyethelene  disk  spacers  occupying  10  percent 
the  length  space.  The  cable  is  assumed  to  be  buried  at  least  four  feet  deep.  At  this  depth,  if 
is  believed  that  yearly  variations  in  temperature  will  almost  never  exceed  ±30°  F.  However, 
the  periods  of  interest  for  the  timing  subsystem  are  daily  variations.  We  will  assume  a daily 
peak  to  peak  temperature  variation  of  0.6°  F.  The  two  major  contributors  to  propagation  delay 
variations  through  the  cable  are  those  due  to  linear  expansion  and  contraction  of  the  len^.h  of 
the  cable  and  changes  in  the  dielectric  constant  both  due  to  temperature.  First,  let  us  consider 
changes  in  the  dielectric  constant  (permittivity)  of  the  polyethelene  spacing  disks  versus  temper- 


ature. 


Propagation  velocity  V is  given  by 


S~e  7 


(7.2.2.1-11 


where  c is  the  free  space  velocity  of  light  and  is  the  relative  dielectric  constant  of  the 
medium.  Let  6 be  the  magnitude  of  the  per  unit  change  in  € r per  unit  change  in  temperature. 


v^r  s/J]+d)er 

t M - -H-) 

Ar  \ /]+ a/ 


(7.2.2. 1-2) 
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Then 


1 

/]  + 6 


(7.2.2. 1-3) 


-5 


For  polyethelene  6 is  approximately  1x10  / C.  Using  this  value  of  <5  we  find 


= 5 x 10""^/°  C.  (7.2.2. 1-4) 

V 

This  is  the  proportional  change  in  the  spacer  medium.  Under  the  assumption  that  only  10  percent 
of  the  total  length  of  the  cable  is  occupied  by  polyethelene  and  the  rest  by  air  and  that  the 
velocity  in  air  is  approximately  equal  to  c then  the  composite  variation  is 

= 0.9  x 0 + 0.1  x 5 x 10-6  = 5 x 10"7/°  C.  (7.2.2. 1-5) 

V 

The  peak  to  peak  daily  proportional  variation  in  velocity  due  to  dielectric  changes  is  then 
approximately  1.6  x 10  7. 

-6  o 

Copper  has  a linear  expansion  coefficient  of  approximately  10x10  / F. 

Then  with  the  assumed  temperature  variation  the  peak  to  peak  cable  length  variation 
A L/L  = 6 x 10"6. 

Thus  we  see  that  the  contribution  to  delay  variation  made  by  dielectric  changes 
can  be  neglected  in  comparison  to  that  made  by  the  length  change  of  the  inner  and  outer 
conductors.  In  the  remainder  of  the  present  derivation  this  will  be  done. 

If  the  length  is  assumed  to  vary  in  a sinusoidal  manner  with  a period  of  one  day, 
then  the  variation  in  length  A L is  given  as: 

4L  = 3xl0^xL  x sin  (cu  t +</>  ) units  (7.2.2. 1-6) 

o dr 

where  L is  the  nominal  length  of  the  cable,  cj  is  the  daily  angular  velocity  = 7.2685  x 10  , 

o a 
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and  <f>  is  some  arbitrary  phase  angle.  The  time  rate  of  change  of  this  length  variation  (speed)  is 


4 L = 3 x 10  xL  x u . cos  (u  t +<t>  ) units/sec 
o d dr 


(7.2. 2. 1-7) 


Letting  V represent  the  nominal  velocity  through  the  cable,  the  Doppler  frequency  shift 
nom 

seen  at  the  receiving  end  of  the  cable  is 

Af  5 fs  I-  $-—)  (7.2.2. 1-8) 

\ nom  / 


where  f is  the  frequency  of  the  reference  source.  Substituting  Equation  (7.2.2. 1-7)  into 
Equation  (7.2.2. 1-8)  we  obtain 


. , 3x10  xL  x u,  cos(cu,t+<£) 

A f o d d r 


(7.2.2. 1-9) 


nom 


7.2. 2. 2 Satellite 

Orbital  variations  in  24  hour  synchronous  artificial  earth  satellites  caused  by  the 

210  209 

earth's  gravity  field  have  been  studied  in  some  detail.  ' Equations  describing  the 
motion  of  the  satellite  with  respect  to  a point  on  the  earth  due  to  this  cause  are  quite  compli- 
cated. Inclusion  of  such  expressions  in  the  simple  model  being  developed  in  this  study  would 
unduly  complicate  this  portion  of  the  model  but  would  produce  no  discemable  enhancement 
to  the  results  of  the  simulation.  On  the  other  hand,  a rough  estimate  of  perturbations  that 
the  timing  network  may  expect  to  see  due  to  satellite  motion  can  be  obtained  from  geometrical 
considerations  of  eccentricity  of  orbit,  maximum  angle  of  inclination  of  plane  of  orbit  with 
respect  to  equatorial  plane,  phasing  of  the  periods  of  motion  due  to  these  two  causes,  and 
location  of  earth  terminals.  A computer  program  has  been  written  at  Harris  ESD  which  takes 
these  factors  into  consideration  and  calculates  maximum  and  minimum  ranges  between  satellite 
and  earth  terminal  as  well  as  maximum  range  rate.  A selection  of  outputs  from  this  program  is 
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shown  in  Table  7. 2. 2. 2 for  a satellite  assumed  to  have  nominal  position  in  the  plane  of  the 
equator  at  0°  longitude  and  earth  stations  at  0°  latitude,  0°  longitude  and  80°  latitude, 

0°  longitude.  Actually,  instead  of  showing  maximum  and  minimum  ranges  the  table  shows 
the  difference  between  the  daily  maximum  and  minimum  range.  Military  satellites  are  expected 
to  have  an  orbital  inclination  of  less  than  3°.^^  |r  ;s  generally  believed  that  eccentricity 
will  be  quite  small  but  values  are  shown  for  0.001,  0.003,  and  0.010  in  Table  7. 2. 2. 2. 

The  terminal  at  80°  latitude  is  at  a very  high  latitude  for  the  applications  under 
consideration,  but  it  appears  to  be  close  to  worst-case.  Thus,  it  appears  that  daily  path 
variation  may  range  from  approximately  zero  at  zero  eccentricity  and  zero  degrees  elevation  up 
to  4,919,000  feet  at  0.01  eccentricity  and  3.0  degrees  elevation  when  the  ground  terminal  is 
at  80°  latitude.  The  maximum  range  rate  may  also  vary  from  zero  to  approximately  180  feet  per 
second.  However,  if  it  is  assumed  that  the  range  difference  varies  in  a sinusoidal  manner  with 
a daily  period  the  model  becomes  extremely  simple.  This  simplifying  assumption  will  be  made 
in  this  study.  Using  a nominal  two  way  range,  L^,  of  248  x 10^  feet  between  the  earth  terminal 
and  satellite,  the  path  length  variation  equation  then  becomes 


A L = K L sin  (u,  t + <t>  ) 
o a r 


where 


0 < K < 3.97  x 10  and 


(j  , - 7.2685  x 10 
d 


Under  these  assumptions  the  daily  fractional  Doppler  frequency  change  is 


A f A L 

~1 V~ 


where  f is  frequency  of  incominq  bit  stream,  4L  is  the  time  derivative  of  A L and  V 

s nom 

is  the  propagation  velocity  between  satellite  and  earth  terminal. 

Due  to  the  large  changes  in  daily  path  length  of  satellite  links  in  relation  to  that 
of  other  transmission  facilities,  it  is  anticipated  that  Doppler  tracking  and  removal  techniques 
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Table  7. 2. 2. 2.  Sample  of  Range  and  Range  Rate  Variations 
Between  Earth  Terminals  and  Satellite 
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may  be  applied  prior  to  obtaining  reference  signals  via  satellite  links  for  the  nodal  timing  sub- 
system. Actually,  the  length  variation  of  a satellite  path  is  quite  predictable  from  knowledge 
of  a particular  satellite's  orbit.  It  is  anticipated  that  this  knowledge  can  be  used  to  advantage 
in  such  a Doppler  removal  device  although  this  has  not  been  investigated  in  this  report. 

7. 2. 2. 3 Microwave 

There  are  several  factors  which  affect  path  delay  variations  of  microwave  links. 
These  factors  include  temperature,  humidity,  atmospheric  pressure,  and  rain  or  fog.  The  first 
three  of  these  terms  can  be  expressed  in  terms  of  "so  called  N units."  The  N unit  is  related 
to  the  index  of  refraction  n (ratio  of  radio  wave  propagation  velocity  in  vacuum  to  that  in  the 
medium)  by 

N = (n  - 1)  106.  (7.2. 2. 3-1 

From  this  value  of  N we  may  obtain  an  equivalent  change  in  path  length  A L, 

4 L = N x 10"6  x L f(t)  (7.2.2. 3-2 

o 

where  L is  nominal  path  length  and  f(t)  is  a function  of  time  which  describes  the  manner  in 
o 

time  in  which  the  variation  takes  place.  The  time  derivative  of  A L may  be  obtained  in  order 
to  calculate  the  amount  of  frequency  Doppler  caused  by  the  delay  variation,  i.e.. 


4L  = Nxl0  xL  x f(t) 
o 


4 f S f 


Daily  temperature  changes  may  account  for  the  most  rapid  fluctuation  in  the  value  of  N.  With 
constant  pressure  and  humidity  a daily  temperature  variation  of  20° C peak  to  peak  about  a 
nominal  20°C  results  in  a change  in  N of  about  18  peak  to  peak.  Largest  monthly  mean 
variation  of  N is  about  ±25  units.  Yearly  temperature  variations  over  the  U.S.  results  in 


1 


. - 


variations  in  N from  230  to  400.  Rain  at  approximately  six  inches  per  hour  results  in  N 
changes  of  about  14  units.  Thus,  for  simulation  purposes,  the  following  will  be  assumed; 

Daily  variations: 

AL  = 10xl0^xL  x sin  ( cu,  t +</>  ) (7.2.2 

o d r 

where  u>  , = 7.2685  rad/sec 
d 

Stress  due  to  rain  or  fog  on  selected  links; 


AL  = 14xRxlO  x 


[M(t  - T)  - p(t  - (T  + D)) 


where  is  a random  phase  between  0 and  2 77-  , 

R is  a random  number  between  0 and  1 , 

D is  a random  number  between  1 ,800  and  43,200  s, 

T is  the  time  in  seconds  at  the  start  of  the  simulation  at  which  the  rain  is 
to  start,  and 


. . 0 for  t < x 

^(t'x)  = 1 for  t > x 


7.2. 2. 4 


Tropospheric  Refraction 


The  troposphere  is  that  area  of  the  atmosphere  extending  from  the  ground  to  a 
height  of  approximately  six  miles.  On  difference  between  the  troposphere  and  the  ionosphere 
is  the  fact  that  there  is  practically  no  ionization  of  the  air  molecules  in  the  troposphere.  The 
troposphere  can  act  as  a smooth  refractive  medium  due  to  variation  with  altitude  of  the 
dielectric  constant  of  the  air.  The  water  vapor  content  of  the  air  will  also  change  the  dielec- 
tric constant.  This  smooth  model  gives  rise  to  beyond  the  horizon  reception  due  to  tropospheric 


refraction . 


Knife  Edge  Diffraction 


According  to  Huygen's  principle,  electromagnetic  energy  is  emitted  by  means  of 
wavelets  that  start  at  every  point  along  the  beam  and  are  transmitted  in  every  direction.  Thus, 
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points  which  lie  in  a region  of  geometric  shadow  may  receive  some  of  the  energy.  Obstructions 
which  give  rise  to  the  knife  edge  effect  are  mountains , hills,  trees  and  the  earth's  horizon.  In 
the  shadow  region  the  beam  is  refracted  by  the  change  in  dielectric  constant  of  the  troposphere. 
This  principle  is  commonly  observed  at  night  when  two  cars  meet  each  other  on  opposite  sides 
of  a hil  I . 


Tropospheric  Scattering 

The  major  contributor  to  dependable  long  distance  communication  (approximately 
300  miles)  is  caused  by  turbulence  and  is  known  as  tropospheric  scattering.  According  to  one 
theory  of  scatter,  the  earth's  atmosphere  is  in  a constant  state  of  mofion.  Because  of  this 
motion  with  respect  to  the  earth,  small  turbulences  or  eddies  are  formed.  These  eddies  are 
similar  to  the  whirlpools  in  a rapidly  moving  stream  of  water.  In  the  atmosphere,  however, 
the  eddies  form  spherical  turbulences  rather  than  the  two-dimensional  whirlpools  in  a stream. 
These  spherical  turbulences  are  called  scattering  blobs  and  are  responsible  for  the  scatter  effect. 
The  turbulent  motion  of  air  at  each  blob  results  in  a variation  in  dielectric  constant  within  the 
blob  and  a small  amount  of  energy  is  scattered  away  from  the  incident  beam.  The  effect  is  the 
same  as  if  each  blob  received  the  signal  and  reradiated  it.  Figure  7.2. 2.4-1  depicts  this 
effect.  The  scattering  blobs  move  through  the  common  volume  of  space  defined  by  transmitting 
and  receiving  antenna  beams  due  mostly  to  the  horizontal  component  of  tropospheric  winds. 

The  geometry  of  this  effect  is  shown  in  Figure  7. 2. 2. 4-2.  The  energy  reaching  the  receiving 
antenna  is  a composite  of  that  gathered  in  from  all  the  scattering  blobs  in  the  common  volume 
of  the  transmitting  and  receiving  beam  space. 

The  problem  at  hand  is  to  determine  the  maximum  path  length  variation  and  the 

rate  at  which  it  may  vary.  The  related  problem  of  frequency  of  signal  strength  fades  has  been 
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discussed  by  Crawford,  et  al.  In  the  extreme  case  when  only  one  scattering  blob  is  in 
the  common  volume,  we  see  from  Figure  7.2.2.4-2B  that  the  maximum  path  length  variation  is 


In  this  same  case  from  ( CW  59  ) the  Doppler  frequency  is 


V sin  5 


(7.2. 2. 4-2) 


Where  A = wave  length  of  the  carrier  signal,  V = horizontal  component  of  tropospheric  wind 
velocity,  5 = angle  between  wind  direction  and  line  Joining  transmitter  and  receiver,  and 
)3  = receiving  and  transmitting  antenna  beam  widths  (assumed  to  be  equal).  However,  the 
model  can  be  simplified  if  a suitable  sinusoidal  motion  is  assumed.  If  it  is  assumed  that  movement 
of  the  scattering  blob  from  center  beam  to  half  power  point  of  the  beam  represents  one-fourth 
cycle  of  the  length  variation  and  a sinusoidal  variation  is  assumed,  then  the  period  of  this 


variation  is 


4 L sin  p/2 
o ^ 


(7.2. 2. 4-3) 


From  Equation  (7. 2. 2. 4-1)  and  Equation  (7. 2. 2. 4-3)  the  maximum  amount  of  path  length 
variation  and  the  minimum  frequency  with  which  the  variations  can  occur  may  be  obtained. 
However,  it  is  believed  that  the  path  length  is  highly  uncorrelated  over  much  shorter  intervals 
of  time  than  the  period  obtained  from  Equation  (7. 2. 2. 4-3)  with  reasonable  values  of  V.  This 
is  consistent  with  numerous  scattering  blobs  being  in  the  common  volume  at  a given  time. 
Nevertheless,  this  equation  will  be  used  to  obtain  an  upper  bound  on  the  period  of  the  path 
length  variation.  On  the  other  hand,  the  frequency  of  the  path  length  variation  is  not  expected 
to  be  as  great  as  that  of  the  highest  frequencies  of  signal  strength  fades.  It  is  believed  that  the 
signal  strength  fades  are  caused  by  multipath  signals  arriving  at  the  receiver  with  phases  of 
such  value  as  to  subtract  from  each  other.  Fromz  signal  strength  fade  frequencies  varies  from 
approximately  .1  Hz  to  10  Hz.  This  frequency  depends  on  antenna  beam  widths.  Narrower 
beamwidths  will  also  reduce  the  peak  amplitude  of  the  length  variation.  The  minimum  period 
of  signal  strength  fades  will  be  taken  as  the  minimum  period  of  path  length  variations.  Using 
LQ  =•  300  miles,  /3  = 1 .8  , and  V = 50  miles  per  hour,  we  obtain  | A L I =195  feet  and 

_3 

co  * 9.26  x 10  rad/sec.  Therefore,  for  purposes  of  simulation,  let  us  assume  that  the  tropo- 

spheric path  length  variation  is  given  as  follows: 
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A L = K L sin(a)  t + <(>  ) 
o x r 


(7.2. 2. 4-4) 


.here  0 < K < 1.23  x 10 

,-3 


-4 


9.26  x 10  < u <2n  x 10^63 

x 


and 


<p  is  a random  phase  angle. 
,-8 


With  K = 1 x 10  and  co  = 3,  peak  proportional  frequency  Doppler  would  be 


A f A L _ 3 x 10  **  iri-13  ./•( 

TT  = V T86000 1.61  x 10  parts/m,  le 

s nom 
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Statistical  Clock  Model 


$■ 

jfc 


>> 

* 


t 
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Three  types  of  timing  clocks  may  be  used  in  the  DCS  network.  These  include  the 

cesium  beam  controlled  oscillator,  the  rubidium  gas  cell  resonator  controlled  oscillator  and  the 

quartz  crystal  oscillator.  The  cesium  beam  frequency  reference  is  a primary  standard.  It  has 

an  intrinsic  frequency  which  can  be  set  up  independent  of  any  other  source  and  has  no  systematic 

drift.  The  rubidium  frequency  standard  is  a secondary  standard.  It  requires  calibration  against 

another  oscillator  and  is  then  subject  to  a small  amount  of  drift.  The  amount  of  drift  is  typically 
-1  -2 

10  to  10  times  that  of  the  best  quartz  crystal  oscillator.  The  quartz  crystal  oscillator  is 
also  a secondary  standard  and  is  utilized  in  those  applications  which  do  not  require  the  extreme 
accuracies  and  stabilities  obtainable  from  the  rubidium  or  cesium  standards.  No  matter  which 
of  these  types  of  clocks  are  utilized  at  any  given  time  at  any  node,  its  essential  properties  may 
be  described  by  a list  of  parameters  which  are  controlled  by  random  number  generators.  The 
parameter  list  should  contain  four  parameters  for  each  nodal  clock.  These  parameters  include 
frequency,  short  term  stability,  accuracy  or  long  term  stability,  and  drift  rate.  Frequency 
refers  to  nominal  frequency.  This  parameter  is  generated  for  each  node  from  a random  generator 
at  the  beginning  of  each  simulation  run  and  is  modified  during  the  run  by  the  accuracy  random 
number  generator  or  by  the  drift  rate  parameter.  At  each  simulation  step,  a short-term  stability 
offset  is  generated  and  added  to  the  current  value  of  nominal  frequency  to  provide  the  mean 
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frequency  for  the  simulation  step.  From  H.P.  literature,  it  appears  that  short  term  stability  (up 

to  approximately  .4  second)  is  essentially  the  same  for  all  three  types  of  frequency  standards  if 

drift  rates  are  not  included  for  rubidium  and  quartz.  H.P.  literature  shows  measurements  for 

square  root  of  the  sum  of  the  squares  of  successive  differences  in  average  frequency  measurements 

-3  +4 

for  measurement  intervals  from  10  to  10  seconds.  Thus,  the  short  term  stability  random 

number  generator  may  sample  from  a normally  distributed  variable  with  zero  mean  and  standard 

8 A f 

deviation  equal  to  the  — (2,  r ) given  by  H.P.  where  r corresponds  to  the  simulation 

step  interval.  These  deviations  are  given  as  piecewise  functions  of  r for  each  clock  type  in 
Table  7.2.3. 


Initial  nominal  frequencies  may  be  selected  from  a normally  distributed  random 

-12 

variate  with  mean  f and  standard  deviation  equal  to  1x10  x f. 


Systematic  drift  or  aging  for  quartz  crystal  oscillators  after  an  initial  operating 
period  of  days  appears  to  be  in  the  range  of  ±1 .57  x 10  ^/sec  but  is  usually  expressed  in 
fractional  parts  per  day,  i.e.,  ±1  x 10  ^/day. 


A similar  systematic  drift  may  be  observed  for  rubidium,  but  it  is  10  to  100  times 
less  than  that  of  quartz.  Drift  rates  for  rubidium  are  usually  expressed  in  fractional  parts  per 
month,  e.g.,  ±1x10  ' Vmonth.  These  drifts  for  quartz  and  rubidium  are  normally  taken  to 
be  the  long  term  stability  of  these  devices.  Cesium  beam  standards  do  not  exhibit  systematic 
drift.  Thus,  a guaranteed  long  term  stability  or  accuracy  over  a range  of  temperature  and 
magnetic  field  intensity  may  be  stated  for  this  standard.  The  fractional  long  term  stability  for 
several  of  H.P.  cesium  beam  standards  is  approximately  1 x 10 


It  should  be  noted  that  the  information  given  in  Table  7.2.3  includes  the  system- 
atic drift  in  the  short  term  stability  data.  Thus,  if  the  short  term  stability  and  long  term  drift 
are  to  be  applied  separately,  then  the  long  term  drift  component  should  be  subtracted  from  the 
data  given  for  values  of  t for  which  the  drift  component  is  an  appreciable  part  of  the  values 
given. 
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Table  7.2.3.  Short  Term  Stability  Standard  Deviations  as  a Function 
of  Sample  Time  (From  HP  Data) 

Quartz 

- 3 s l°8io  t s . .55  . lOgl0  a = - logl0  T - 11.7049 

- .55  < logi0  t s 4 . o = 7 x 10  12  or  log1Q  a = - 11.1549 

Systematic  Drift  ± 1 x 10’10/day  -*  1.5 7 x lo'15/sec 

Rubidium 
-**  • 

" 3 5 lo8i0  T < - 1 , loglQ  a = - lOg10  T - ll.  7049 

1 < loSio  t • 2 , log10  ct  = - . 5 log j0  T - 11.1549 

2 < lo8io  T < 4 , a = 7 x 10  13  or  log1Q  0 = - 12.1549 

Systematic  Drift  ± 1 x lO'U/month  3. 858  x 10‘18/sec 


Ce  8ium 


' 3 * loB i o T « - .55  , 

lo8io  c = - log10  T - 11.  7049 

- . 55  <•  log10  t < . 6 

a = 7 x 10  12  or  logl0  a = - 11.1549 

• <>  * l°8io  t < 4 

o = - . 5 log10  t - 11.  04 

Systematic  Drift  - None  , 

I-ong  Term  Stability  ± 1 x 10 

Simulation  Results 


7.3.1  The  Simulation  Plan 

A plan  was  drawn  up  at  the  midpoint  of  the  study  whereby  the  simulator  could  be 
used  to  test  the  sensitivity  of  the  master-slave,  mutual  sync  and  TRD  techniques  to  various  dis- 
turbances and  stresses.  Since  initial  startup  with  a set  of  random  phase  errors  and  frequency 
offsets  generates  possibly  as  severe  a transient  as  the  network  will  ever  see,  it  was  decided  that 
a set  of  runs  for  each  of  the  network  configurations  should  be  made  with  only  the  initial  condi- 
tions applied.  Next  the  same  set  of  runs  were  to  be  repeated  but  this  time  with  the  norma!  link 
delay  variations  applied.  A third  set  of  runs  were  then  to  be  made  with  initial  conditions,  normal 
link  delay  variations  and  stress  events  applied.  Finally,  a set  of  runs  were  to  be  made  with  one 
of  the  network  configurations  for  each  of  the  timing  techniques  using  identical  initial  conditions, 
link  delay  variations,  stress  events  and  duration  of  run  in  order  to  get  a direct  comparison  of  the 
different  techniques  under  the  same  network  configuration  and  inputs.  The  objective  of  the  first 
three  sets  of  runs  was  to  test  for  sensitivity  as  a function  of  network  topology,  as  a function  of 
place  of  origin  in  the  network  at  which  a stress  originates,  i .e . , near  the  connectivity  center  of 
the  network  as  opposed  to  the  fringe  of  the  network  and  as  a function  of  the  type  of  disturbance, 
e.g.,  change  in  clock  frequency  or  phase,  change  in  link  delay,  loss  of  link  and  loss  of  node. 

In  order  to  access  these  sensitivities,  the  plan  called  for  observation  of  peak  frequency  offsets, 
phase  errors,  propagation  distance  of  disturbance  from  place  of  origin  measured  in  nodes,  and 
how  long  it  takes  initial  and  transient  conditions  caused  by  the  various  stresses  to  die  out. 

Some  of  the  above  parameters  were  actually  analyzed  under  more  carefully  con- 
trolled conditions  with  limited  network  structures  in  the  discussion  on  error  propagation  in  Section 
8.0.  However,  the  overall  objective  of  the  simulation  plan  was  to  see  how  the  larger  network 
behaves . 

The  plan  proved  to  be  overly  ambitious  in  that  all  the  simulations  implied  by  the 
plan  could  not  be  obtained  and  completely  analyzed  in  the  limited  time  available,  however  some 
very  useful  information  concerning  the  behavior  of  the  timing  techniques  under  what  is  believed 
to  be  real  world  conditions  was  obtained  via  the  simulation  effort.  In  working  with  the  simulator, 
it  was  found  that  perhaps  the  true  value  of  a simulation  lies  not  so  much  in  providing  answers  to 
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preconceived  specific  questions  but  rather  more  in  causing  the  user  to  ponder  his  situation  in 
sufficient  detail  that  he,  of  necessity,  discovers  what  the  real  questions  are. 
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7.3.2  Network  Configurations 

The  four  network  configurations  used  in  the  simulations  are  shown  in  Figures  7. 3. 2-1 
through  7. 3. 2-4.  When  referring  to  these  figures  in  the  remainder  of  this  section,  they  will  be 
referred  to  simply  by  their  alphabetical  symbols,  included  on  the  figures.  The  overall  charac- 
teristics of  these  network  configurations  are  discussed  in  Section  5.0.  The  initial  timing  hierarchy 
for  master-slave  and  TRD  are  indicated  by  arrowheads.  The  initial  master  node  is  denoted  by 
the  letter  M above  the  node  number.  The  links  are  marked  according  to  type  and  distance. 

The  nomenclature  is  M for  line  of  sight  microwave,  C for  coaxial  cable,  S for  satellite  and  T 
for  beyond  the  horizon  tropospheric.  The  distances  are  in  kilometers.  Thus  the  Label  C 660 
indicates  a cable  link  of  distance  660  kilometers.  Due  to  the  limited  storage  availability  on  the 
Datacraft  computer  used  in  support  of  the  simulation  effort,  all  nodes  and  links  could  not  be 
monitored.  When  monitoring  a mutual  sync  node,  all  the  incoming  links  to  this  node  must  be 
monitored.  Thus,  only  a subset  of  the  nodes  were  monitored.  These  are  marked  with  a for 
master  slave  and  TRD  and  with  a "+"  for  mutual  sync.  The  monitored  nodes  were  composed  mostly 
of  the  backbone  of  the  network  with  a few  other  nodes  at  Levels  2 and  3. 

7.3.3  Loop  Parameters 

The  values  used  in  the  simulations  for  nodal  loop  filter  parameters  were  the  same  as 
those  derived  in  Section  3.0. 

7.3.4  Input  Signals 

7. 3. 4.1  Initial  Conditions 

Most  of  the  nodal  clock  initial  frequency  offsets  were  set  to  zero,  but  a few  nodes 

were  chosen  at  random  and  assigned  an  offset  from  a normal  distribution  with  zero  mean  and 
-8 

±3^  ±1x10  x fQ.  The  initial  phase  ^ at  the  receiving  end  of  links  was  computed  as 

fol  lows: 
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Figure  7. 3. 2-1.  Dumbbell  Node  Configuration  (Configuration  A1 


Figure  7. 3. 2-2.  Meandering  Node  Configuration  (Configuration  B) 


Figure  7. 3. 2-3.  Ladder  Node  Configuration  (Configuration  D) 


Figure  7.3.2 


<^>1  = (ND  4 7T ) mod  27 T - 77 


where  ND  in  the  nominal  link  delay  in  radians  of  the  basic  clock  frequency,  fQ.  The  initial 
value  of  nominal  link  disturbance  is  then  added  to  this  value. 

7. 3. 4. 2 Normal  Link  Disturbances 

Normal  link  disturbances  of  the  maximum  amplitude  as  discussed  in  Paragraph  7.2.2 
were  applied  to  all  links  except  tropospheric  variations  were  not  applied.  The  tropospheric  model 
indicates  that  the  frequencies  of  the  disturbances  are  sufficiently  high  that  they  would  be  attenu- 
ated enough  by  loop  filters  to  be  unnoticable  due  to  scale  resolution  of  the  output  plots.  Link 
asymmetries  for  TRD  runs  were  adjusted  to  1 percent  of  peak  delay  variation. 

7. 3. 4. 3 Clock  Drift  and  Short  Term  Jitter 

Clock  drifts  and  jitter  were  not  applied.  With  quartz  clocks  drifting  in  frequency 

at  a rate  of  ±1  x 10_1°  x f per  day,  the  total  amount  of  drift  would  not  be  great  enough  to  cause 

appreciable  error  if  neglected.  As  indicated  in  Section  3.0  the  Type  2 loop  tracks  a constant 

2 

drift  of  0 Hz/s  with  a phase  error  of  0/a»n  . The  tracking  loop  parameters  utilized  with  the 
master-slave  configuration  will  produce  a constant  phase  offset  of  0.39  ps  with  0=10  x f^ 
per  day.  For  the  Type  1 filter,  constant  drift  will  produce,  in  steady-state,  a constantly  increas- 
ing phase  error  of  the  form 


-3  -10 

For  the  mutual  sync  nodes  using  ^ = 1 , tJn  = 1 .67  x 10  and  0-10  x fQ  Per  ^ay  *^'s  9'ves 

• <p  = 0.1 17  ps  accumulated  per  day. 

e 

Since  simulation  runs  were  limited  to  approximately  3 days  or  less,  drift  terms  were  discarded. 

1 

( The  clock  models  indicate  that  short  term  clock  jitter  will  be  filtered  sufficiently  by  loop  filters 

that  it  will  be  of  no  consequence  in  the  simulation  results. 
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7.3.5 


Stress  Events 


The  stress  events  for  master-slave  and  TRD  runs  are  shown  in  Figures  7. 3. 5-1  through 
7. 3. 5-4.  The  events  for  mutual  sync  ere  shown  in  Figures  7. 3. 5-5  through  7. 3. 5-8.  In  addition, 
master-siave  and  TRD  also  utilize  the  stress  event  of  Figure  7. 3. 5-9.  In  observing  the  results  of 
the  stress  events,  it  should  be  i.ept  in  mind  that  the  individual  r.od'-'  frequency  response  to  step 
changes  in  reference  frequency  or  phase  error  will  have  shapes  like  that  shown  in  Figure  A6  and 
Figure  A7  of  Appendix  A.  The  phase  plots  will  show  responses  to  step  rhr».  .s  in  reference  fre- 
quency and  phase  error  similar  to  those  shown  in  Figure  A8  ar.d  Figjie  A9  ol  ■ pDendix  A.  How- 
ever, the  curves  become  more  difficub  to  evalucre  when  .iiulople  forcing  func.'Ons  combine  to 
cause  a composite  time  response  from  the  connected  network. 


EVENT  TIME  (s) 


EVENT 


Node  15  fa i Is 

Node  9 slaves  to  Node  13 

Node  17  fails 

Node  21  slaves  to  Node  19 
Node  24  slaves  to  Node  25 
Node  18  fails 

Node  21  slaves  to  Node  24 

Node  19  slaves  to  Node  21 

Node  20  slaves  to  Node  21 

Node  25 's  clock  frequency  changes 

by  +1  x 1 0 ^ x f 
' o 


EVENT  TIME  (s) 


EVENT 


16000 

18000 

20000 

22000 

23000 


Link  (18-14)  fails 

Node  8 slaves  to  Node  18 

Node  14  slaves  to  Node  8 

Node  18  clock  frequency  changes  by 

+1  x 10“8  x f 

o 

Node  14  free  runs 

Node  8 slaves  to  Node  14 

Node  18  slaves  to  Node  8 

Link  (18-23)  fails 

Node  23  slaves  to  Node  20 


i 


I 
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EVENT  TIME  (s) 
10000 
11000 
14000 


22000 


EVENT 
Node  8 fails 
Node  20  fails 

Node  21  slaves  to  Node  19 
Node  24  slaves  to  Node  21 
Node  7 slaves  to  Node  22 
Node  23  slaves  to  Node  22 
Node  5 slaves  to  Node  7 
Node  6 slaves  to  Node  5 
Node  16  slaves  to  Node  6 
Node  19  slaves  to  Node  16 


Figure  7. 3. 5-3.  Stress  Events  for  Master-Slave  and  TRD;  Configuration  D 
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EVENT  TIME  (s) 
11000 
12000 

13000 

16000 

18000 

20000 


EVENT 

Link  (10-14)  fails 
Node  15  slaves  to  Node  12 
Node  14  slaves  to  Node  15 
Node  14  fails 

Node  18  slaves  to  Node  19 
Node  10  fails 

Node  19  slaves  to  Node  18 
Node  13  slaves  to  Node  16 
Node  1 1 slaves  to  Node  13 
Node  8 slaves  to  Node  1 1 
Node  2 slaves  to  Node  4 
Node  3 slaves  to  Node  2 
Node  6 slaves  to  Node  3 
Node  15  slaves  to  Node  19 
Node  12  slaves  to  Node  15 
Node  9 slaves  to  Node  12 


Figure  7. 3. 5-4.  Stress  Events  for  Master-Slave  and  TRD;  Configuration  E 
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EVENT  TIME  (s) 


EVENT 


Clocks  of  Nodes  18,  25,  and  21  experience 
a step  increase  in  natural  frequency  of 


1x10  xf 


20000 


Clocks  at  Nodes  15,  13,  and  14  experience 


a step  decrease  in  natural  frequency  of 


1x10  xf 


40000 


Link  (16-18)  fails 


60000 


Link  (17-18)  fails 
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Figure  7. 3. 5-5.  Stress  Events  for  Mutual  Sync;  Configuration  A 
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Node  20  is  restored  and  Nodes  7,  8,  19,  21, 
22,  23,  and  24  place  Node  20  into  their 
reference  signal 

Node  4 fails  and  Nodes  1,  2,  3,  6,  and  16 
remove  Node  4 from  their  weighting  functions 
Node  4 is  restored  and  Nodes  1,  2,  3,  6 and 
16  replace  Node  4 back  into  their  reference 
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Links  (2-4),  (4-6),  (6-8),  (8-20),  (20-24), 
(24-28),  (28-30)  experience  a ramp  increase 
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Delay  in  above  links  becomes  stationary 


Figure  7. 3. 5-7.  Stress  Events  for  Mutual  Sync;  Configuration  D 
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EVENT  TIME  (s) 


WENT 
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10000  Node  22  fails  and  Nodes  23,  25,  26,  and  27 

remove  Node  22  from  their  reference  signals 
20000  Node  22  is  restored  and  Nodes  23,  25,  26, 

and  29  replace  Node  22  back  into  their 
reference  signals 

40000  Node  10  fails  and  Nodes  2,  6,  8,  9,  12,  and 

14  remove  Node  10  from  their  reference  signals 
50000  Node  10  is  restored  and  Nodes  2,  6,  8,  9,  12, 

and  14  replace  Node  10  back  into  their 
reference  signals 

80000  Node  1 4's  clock  experience  a step  increase  of 

-8 

1 x 10  x f in  natural  frequency 
o 

120000  Links  (8-10),  (10-14),  (14-18),  and  (18-25) 

experience  a ramp  increase  in  delay  of 

1 x 10  7 x D Aour 
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160000  The  delay  of  the  above  links  become  stationary 
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Figure  7. 3. 5-9.  Comparison  Stress  Events  for  Master-Slave  and  TRD  for  Configuration  B 
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7.3.6 


Simulation  Run  Results 


Although  all  the  runs  implied  by  the  simulation  plan  were  not  obtained,  con- 
siderable more  result  data  was  produced  than  could  be  included  in  this  report  in  a comprehen- 
sible form.  Therefore,  runs  that  appear  to  be  representative  of  the  timing  techniques,  network 
configurations,  disturbances  and  stress  events  were  selected.  From  each  of  these  selected  runs 
link  to  node  phase  errors  and  nodal  frequency  errors  for  a few  of  the  links  and  nodes  experi- 
encing perturbations  most  representative  of  the  general  network  behavior,  or  which  could  serve 
to  illustrate  particular  relationships  between  stimulus  and  response  were  plotted.  Sets  of  runs  were 
actually  made  consisting  of  initial  transient  only  (IT),  initial  transient  plus  normal  link  disturbances 
(IT  + ND),  and  initial  transient  plus  normal  link  disturbances  plus  stress  events  (IT  + ND  + SE),  but  in 
order  to  reduce  the  volume  of  material  onlythe  IT  + ND  + SE  runs  from  each  set  are  shown  in  this 
report.  The  entire  mutual  sync  networks  were  perturbed  by  normal  satellite  link  disturbances 
but  the  effect  was  mostly  swamped  by  the  response  to  stress  events  as  shown  in  the  (IT  + ND  + SE) 
plots  so  the  amplitudes  of  the  phase  and  frequency  errors  due  to  normal  link  disturbance  are  shown 
in  tabular  form  for  mutual  sync  runs  involving  networks  with  satellite  links. 


Mutual  Sync 


Configuration  A - Normal  Link  Variations 

The  loop  time  constants  for  the  mutual  sync  nodes  were  chosen  considerably  shorter 

than  those  for  the  tracking  mode  of  the  master/slave  or  TRD  techniques  in  order  to  provide  a 

comparable  response  time  on  a node  for  node  basis  to  transient  conditions.  The  resulting  magnitude 

of  nodal  gain  for  daily  path  delay  variations  turned  out  to  be  1 .0019.  Thus,  these  variations 

passed  through  each  node  without  attenuation.  These  variations  can  be  observed  on  the  initial 

transient  plus  normal  link  disturbance  plots.  There  was  only  one  satellite  link,  (I4-16C),  in 

this  network  configuration.  This  link  was  responsible  for  almost  all  of  the  daily  variation  in  the 

network.  The  peak  variation  on  the  link  was  1 .2  x 10_1°  s/km.  The  length  of  the  link  was  set 

at  96,800  km.  This  gives  a peak  variation  of  11.62  x 10  6second.  Since  the  variation  was 

assumed  to  be  sinusoidal  with  a period  of  1 day,  the  resulting  peak  frequency  doppler  was 
A f 

— = 8.5  x 10  . Some  of  the  nodes  peak-to-peak  frequency  variations  were  as  follows: 


I 


Node 


1 

Composite  Plots 

Figures  7. 3. 6. 1-1  and  7. 3. 6. 1-2  show  composite  initial  transient,  normal  link 
disturbance  and  stress  event  plots  of  link  to  node  phase  errors  for  Links  14-15,  16-17,  and 
16-25  and  nodal  frequency  errors  for  Nodes  15,17,  and  25. 

-8 

The  step  change  in  Nodes  18,21  and  25's  frequencies  of  1 x 10  x f^  at  time 

10.000  seconds  caused  Node  17's  clock  to  change  by  32  x 10  10  x f but  had  little  effect  on 

-8  ° 

the  nodes  in  rite  left  half  of  the  network.  The  change  of  1 x 10  x f in  clocks  13,  14,  and  15 

at  20,000  seconds  had  little  effect  on  the  nodes  of  the  right  half  of  the  network.  Node  16  acted 

as  a buffer  during  the  period  from  10,000  seconds  to  20,000  seconds.  The  net  result  of  these  two 

changes  was  almost  zero  by  40,000  seconds.  At  40,000  seconds  when  Link  16-18  failed  Node 

1 5's  clock  decreased  in  frequency  by  approximately  27.6  x 1 0 xf  . Loss  of  Link  1 7-1 8 at 

° -10 

60.000  seconds  caused  Node  17's  clock  frequency  to  decreased  by  about  1 7 x 10  x f . The 

increase  in  delay  on  Link  14-16  at  time  100,000  seconds  caused  almost  no  change  in  any  of  the 

three  plotted  nodes'  frequencies.  When  Link  15-16  failed  at  time  140,000  seconds,  Node  15 

decreased  in  frequency  by  approximately  73  x 10  x f ■ Failure  of  Node  17  at  180,000  seconds 

° -10  , 

caused  Node  1 5's  clock  frequency  to  decrease  by  approximately  17x10  x f and  Node  25's 
by  approximately  11  x 10  ^ x f . The  smaller  change  in  Node  25's  frequency  is  attributed  to 
the  larger  number  of  nodes  connect  to  25.  Failure  of  Node  24  at  220,000  seconds  caused  25's 
frequency  to  momentarily  decrease  by  approximately  32  x 10  x fQ. 

Configuration  B,  Normal  Link  Variations 

Although  this  configuration  shows  three  satellite  links,  the  simulator  only  utilized 
the  two  satellite  Links  8-18  and  8-27  . When  the  input  was  generated  for  this  simulation 
run  the  satellite  link  between  Nodes  18  and  27  was  inadvertently  replaced  by  a cable  link  of 
length  660  km.  Nevertheless,  the  two  satellite  links  were  the  source  of  most  of  the  daily 
variation  seen  through  the  network.  The  daily  peak-to-peak  frequency  and  phase  variations 
observed  are  as  follows: 
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Figure  7. 3. 6. 1-1  . Mutual  Sync,  Configuration  A,  IT  + ND  + SE  Frequency  Plots 


igure  7.C 


Figures  7.3.6. 1-3  and  7. 3. 6. 1-4  show  composite  initial  transient,  link  disturbance 
and  stress  events  plots  of  link  to  node  phase  errors  for  Links  8-14,  18-23,  27-8,  and  27-29  and 
nodal  frequency  errors  for  Nodes  8,  14,  23,  and  29.  » 


The  initial  frequency  transient  at  Node  8 is  caused  mostly  by  the  combined  initial 
frequency  error  at  Node  1 0 of  26  x 1 0 ^ x f and  initial  phase  errors  between  Node  8 and  the 
satellite  links.  Each  of  the  nodes  plotted  experienced  a daily  frequency  variation  of  approxi- 
mately 9x10  ^ x f due  to  satellite  links  between  Nodes  8 and  18  and  between  Nodes  27  and 
' o 

8.  The  link  delay  change  on  Link  8-18  at  time  20,000  seconds  was  not  noticeable  except  at 

Node  8 where  it  appears  to  have  caused  an  increase  in  frequency  of  approximately  2x10  x f . 

The  failure  of  Node  5 at  40,000  seconds  caused  a change  in  Node  8 1 s frequency  of  approximately 

4.8  x 10  ^ x fQ.  Failure  of  Link  23-24  at  80,000  seconds  caused  a change  in  Node  23's  frequency 
“10  “8 

of  16.25  x 10  u x f . The  change  of  1 x 10  x f in  Node  1 8* s nominal  clock  frequency  at 

° ° -10 
120,000  seconds  caused  Node  23's  frequency  to  change  by  approximately  26  x 10  x f and 

-10  ..  . 0 

Nodes  8 and  29  by  approximately  11x10  x f . Failure  of  Link  8-18  at  18,000  seconds  caused 
a change  in  Node  8's  clock  frequency  of  15.6  x 10  ^ x f and  approximately  2.6  x 10  ^ x fQ 
in  those  of  Nodes  14  and  29.  The  change  in  Node  18's  nominal  clock  frequency  at  120,000 
seconds  appears  to  have  changed  the  system  frequency  by  9 x 10  ^ x f . Although  the  link 
failures  caused  momentary  transients  in  particular  nodal  frequencies  whose  amplitude  were 
comparable  to  those  caused  by  a change  in  nominal  clock  frequency  of  1 x 10  x f the  link 
failures  do  not  appear  to  have  caused  a change  in  system  frequency. 


OCl 


Figure  7.3.6. 1-3.  Mutual  Sync,  Configuration  B,  IT  + ND  + SE  Frequency  Plots 


Figure  7.3.6. 1-4.  Mutual  Sync,  Configuration  B,  IT  + ND  + SE  Frequency  Plots 


7. 3.6. 2 


Master-Slave 


Under  the  master-slave  scheme,  each  node  is  slaved  to  the  timing  received  via  the 
communications  link.  Thus,  it  sees  all  link  perturbations  in  this  reference  information.  However, 
by  making  individual  node's  loop  time  constants  long  enough,  these  perturbations  can  be  filtered 
sufficiently  that  only  the  long  term  average  reaches  the  VCO  control  signal. 
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7. 3. 6. 2.1  Configuration  A 

Figures  7. 3. 6. 2. 1-1  and  7. 3. 2. 6. 1-2  shows  composite  initial  transient  - normal 
link  disturbance  - stress  events  plots  of  link  to  node  phase  errors  for  Links  14-15,  16-17,  16-25, 
and  25-29  and  noda  I frequency  errors  for  Nodes  15,  17,  25  and  29. 

This  run  shows  a transient  and  a step  change  in  the  natural  frequency  of  a clock  in 

the  timing  chain.  Node  25  starts  at  time  0 with  a frequency  error  of  approximately  -2.7  x 10  ^ 

x f and  a phase  error  with  respect  to  its  reference  via  Link  16-25  of  approximately  -0.47  ps. 

These  errors  are  cancelled  by  6000  seconds.  The  other  nodes  experience  similar  initial  transients. 

The  time  scale  is  broken  at  6000  seconds  and  picks  back  up  at  30,000  seconds.  At  31,100  seconds 

_0 

Node  25's  clock  natural  frequency  makes  a step  increase  of  1 x 10  x f . At  31,300  seconds 
Node  25  has  built  up  enough  phase  error  with  respect  to  its  reference  via  Link  16-25  to  trigger 

into  its  acquisition  mode.  This  causes  a frequency  jump  from  approximately  f (1  +91  x 10 

-10  ° 
to  f (1  - 91  x 10  ) at  31,300  seconds.  The  accumulated  phase  error  and  frequency  error  is 

o 

then  reduced  to  close  to  zero  by  35,000  seconds.  At  31,100  seconds  Node  29  begins  to  follow 
Node  25  up  in  frequency  but  the  phase  error  does  not  quite  reach  a high  enough  value  at  31 ,300 
seconds  to  trigger  into  acquisition  mode.  The  errors  at  Node  29  are  also  reduced  to  zero  between 
31 ,300  and  35,000  seconds.  The  phase  error  of  approximately  0.4  ps  associated  with  Link  14-15 
is  between  two  communicating  nodes  in  separate  timing  chains  with  each  node  one  level  from  the 
master . 

7. 3. 6. 2. 2 Configuration  D 

Figures  7. 3. 6. 2. 2-1  and  7. 3. 6. 2. 2-2  show  composite  initial  transient  - normal 
link  disturbance  - stress  events  plots  for  link  to  node  phase  errors  for  Links  5-7,  6-16,  22-23, 
and  24-28  and  nodal  frequency  error  for  Nodes  7,  16,  23,  and  28. 
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gure  7. 3. 6. 2. 1-1.  Master-Slave,  Configuration  A,  IT  + ND  + SE  Frequency  Plots 


+ SE  Phase  Plots 


Figure  7. 3. 6. 2. 2-2.  Master-Slave  Configuration  D,  Phase  Plots  IT  + ND  + SE 


This  run  shows  an  initial  transient  plus  a major  network  reorganization.  The  initial 
transient  is  only  shown  out  to  4000  seconds.  The  time  scale  then  resumes  at  14,000  seconds.  The 
vertical  scale  of  the  frequency  plot  is  also  broken  below  -6.5  parts  in  10^  of  f and  picks  up 
again  at  -130  parts.  The  initial  transient  shows  that  all  nodes  begin  with  initial  frequency 
errors  of  less  than  1 .8  parts  in  10^  and  are  close  to  zero  by  4000  seconds.  The  phase  errors 
shown  except  for  24-28  are  with  respect  to  nodes  in  different  timing  chains  during  the  initial 
transient  and  do  not  appear  to  have  settled  out  by  4000  seconds,  but  errors  are  all  less  than 
0.5  ps.  At  10,000  seconds  Node  8 fails.  Node  20  fails  at  1 1 ,000  seconds.  The  first  part  of 
reorganization  then  begins  at  14,000  seconds  when  Node  21  slaves  to  19,  Node  24  slaves  to  21, 
Node  7 slaves  to  22  and  Node  23  slaves  to  22.  This  causes  a step  change  in  Node23's  frequency 
of +2 .93  parts  in  10^.  Nodes  24  and  7 experience  smal ler  steps . The  plot  shows  Node  16  with 
approximately  +2.3  parts  in  10^  of  error  at  14,000  seconds  but  the  difference  between  its 
initial  value  and  this  value  is  only  the  smallest  increment  of  the  computer  output.  The  second 
part  of  the  reorganization  occurred  at  22,000  seconds  when  Node  5 slaves  to  7,  Node  6 to  5, 

Node  16  to  6 and  19  to  16.  This  left  the  remaining  network  completely  linked  in  a timing  sense 
with  Node  22,  the  new  network  master.  Of  those  nodes  shown  only  Node  16  changed  references 
at  this  time.  Since  Node  16  had  a phase  error  with  respect  to  the  new  reference  signal  via  Link 
6-16  of  greater  than  -2  ps  when  the  switch  occurred  it  immediately  went  into  its  acquisition 
mode.  This  caused  its  frequency  error  to  jump  from  +2.3  parts  in  10^  to  -194  parts  in  10^. 

The  errors  at  this  node  were  then  rapidly  reduced  during  the  period  22,000  to  26,000  seconds. 


7. 3. 6. 2. 3 Configuration  E 

Figures  7. 3. 6. 2. 3-1  and  7. 3. 6. 2. 3-2  show  composite  initial  transient  - normal 
link  disturbance  - stress  events  plots  for  link  to  node  phase  errors  10-8,  14-15,  14-18  and 

f 

18-25  and  nodal  frequency  errors  for  Nodes 8,  15,  18  and  25. 

This  run  shows  an  initial  transient,  a link  failure,  two  node  failures  and  subsequent 
reorganization  following  these  failures.  The  initial  transient  is  shown  out  to  4000  seconds  after 
which  the  time  scale  is  broken.  The  peak  frequency  errors  of  the  nodes  shown  during  the  initial 
transient  were  less  than  3 parts  in  10^.  Peak  phase  errors  were  all  less  than  0.6  ps.  All 
depicted  nodes  had  reached  approximately  steady  state  conditions  with  close  to  zero  frequency 
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Figure  7. 3. 6. 2. 3-1.  Master-Slave,  Configuration  E,  Frequency  Plots,  IT  + ND  + SE 


and  phase  errors  by  12,000  seconds.  At  11,000  seconds  Link  10-14  failed  and  Node  14  coasted 

without  reference  until  12,000  seconds  at  which  time  Node  15  changed  reference  from  14  to  12 

and  Node  14  began  referencing  Node  15.  This  reorganization  caused  a step  change  in  Node  1 5's 

frequency  of  approximately  -1.4x10  x f . Nodes  18  and  25  followed  Node  15  via  Node  14. 

At  13,000  seconds  Node  14  failed  and  Node  18  coasted  until  16,000  seconds  at  which  time  it 

began  to  reference  Node  19.  This  caused  a step  change  in  Node  18's  frequency  of  approximately 

-3.25  x 1010  x f au?  to  the  phase  error  between  Node  1 8 1 s clock  and  that  at  the  receiving  end 
o 

of  Link  19-18  of  approximately  0.15  ps.  Node  25  began  to  follow  this  change.  At  18,000 

seconds  Node  10  failed  and  Node  12  began  coasting.  At  20,000  seconds  Node  19  began  to 

reference  Node  18  with  18  still  referencing  19.  Nodes  18  and  19  then  referenced  each  other 

for  the  remainder  of  the  run.  Also  at  20,000  seconds  much  of  the  remainder  of  the  network 

changed  references  but  the  changes  of  most  interest  were  that  Node  15  referenced  19,  13 

referenced  16,  11  referenced  13  and  8 referenced  11  . This  caused  a step  change  in  Node  1 5 1 s 

frequency  of  approximately  +0.65  x 10  ^ x f after  which  it  then  approached  Node  1 9 1 s fre- 

° -10 

quency.  Node  8's  frequency  changed  by  approximately  -1.3x10  x f . Since  Node  8 was 
then  four  nodes  away  from  its  ultimate  reference  and  this  entire  chain  was  undergoing  a transient 
it  was  still  heading  away  from  its  ultimate  reference's  frequency  at  the  end  of  the  simulation. 
After  a node  fails,  the  simulator  stops  updating  the  nodal  frequency,  i.e.,  the  nodal  frequency 

t' 

remains  at  the  last  value  before  the  failure.  Thus,  phase  errors  may  increase  rapidly  as  shown 
for  1 4-1 5 and  1 4-1  8 after  1 3,000  seconds  or  10-8  after  1 8,000  seconds. 


7. 3. 6. 3 Time  Reference  Distribution 


It  should  be  noted  that  what  is  plotted  as  phase  is  the  phase  error  between  the 
local  node  and  the  received  phase.  The  nominal  delay  of  the  link  is  taken  mode  rrand  then  any 
delay  variation  and/or  difference  between  the  reference  node's  clock  phase  and  the  local  clock 
phase  is  added  to  this  module  value  to  obtain  the  phase  values  that  are  plotted.  This  value  is 
actually  the  error  signal  of  the  phase-locked  loop  in  the  master-slave  scheme.  On  the  other  hand, 
the  TRD  scheme  tends  to  adjust  each  local  clock's  phase  so  that  it  corresponds  to  the  phase  of 
the  master  clock  without  regard  for  link  delays  or  variations  thereto.  Thus,  there  is  no  fixed 
relationship  between  the  error  signal  which  is  fed  into  each  of  TRD  node's  phase-locked  loops  and 
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the  values  that  are  plotted  as  phase.  Thus,  the  phase  plots  for  the  TRD  scheme  will  not  in  general 
be  zero  in  the  steady  state.  One  consequence  of  this  is  that,  to  the  degree  that  the  TRD  scheme 
accomplishes  its  purpose,  the  total  link  perturbations  should  show  up  in  ihe  TRD  phase  plots. 
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7. 3. 6. 3.1  Configuration  A 

Figures  7. 3. 6. 3. 1-1  and  7. 3. 6. 3. 1-2  show  composite  initial  transient  - normal 
link  disturbance  - stress  events  plots  for  link  to  node  phase  errors  for  Links  19-13,  16-17, 

16-25,  18-19,  and  25-29  and  noda  I frequency  errors  for  Nodes  13,  17,  25,  19  and  29. 

This  run  shows  an  initial  transient  and  a step  change  in  natural  frequency  of  one 

clock  in  the  network.  Other  events  occurred  but  no  perturbations  were  observed  on  the  links 

or  nodes  which  are  shown.  Due  to  an  initial  frequency  error  at  Node  13,  a phase  error  sufficient 

to  trigger  into  the  acquisition  mode  (2  ps)  is  acquired  by  1300  seconds  at  which  time  Node  13 

switched  to  its  acquisition  mode.  This  resulted  in  a step  change  inNodel3's  frequency  of 

-111  x 10  ^ x fQ.  These  errors  then  quickly  return  to  close  to  zero  by  2200  seconds.  Similarly, 

Node  17  had  an  initial  frequency  error  of  approximately  +12  x 10  ^x  f . Consequently,  it 

o 

built  up  enough  phase  error  by  2450  seconds  to  switch  to  its  acquisition  mode.  This  caused  a 

step  change  in  Node  1 7's  frequency  of  -150  x 10  ^ x f . The  reason  for  the  difference  between 

these  two  changes  in  the  graph  is  that  the  step  size  of  the  computer  output  was  too  coarse  to 

catch  the  actual  peaks . At  31 , 1 00  seconds  Node  25's  clock  made  a step  increase  in  frequency 

of  1 x 10  10x  f . At  31,300  seconds  the  phase  error  between  Node  25  and  the  ultimate  reference 
o 

reached  a sufficient  value  (2  ps)  to  cause  Node  25  to  switch  to  its  acquisition  mode.  This  caused 
its  frequency  to  change  by  approximately  175  x 10  ^ x fQ  to  an  error  of  -85  x 10  ^ x f . 

These  errors  were  then  reduced  to  approximately  zero  by  time  32,000  seconds.  Node  29  which 
was  slaved  to  25  saw  the  sum  of  the  error  between  i tself  and  Node  25  and  the  error  between  Node 
25  and  the  master  node.  This  would  be  zero  during  the  entire  transient  except  that  the  informa- 
tion that  it  obtains  about  the  error  between  Node  25  and  Node  16  is  one  sample  period  older  than 
that  between  itself  and  Node  25.  Thus,  it  saw  a small  error  signal  and  remained  fairly  close  in 
phase  and  frequency  to  Node  16,  the  ultimate  reference.  Table  7. 3. 6. 3.1  lists  the  times  at  which 
the  various  nodes  made  transitions  from  tracking  to  acquisition  mode  and  vice  versa.  Nodes  21 , 

24  and  30  were  not  monitored. 
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Figure  7. 3. 6. 3. 1-1 . TRD  Configuration  A,  Frequency  Plots,  IT  + ND  + SE 
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7.3. 6. 4 


Simulotion  Run  Results  Summary 


A tabular  summary  of  the  simulation  run  results  is  shown  in  Table  7. 3. 6. 4. 

A link  dropout  or  node  failure  results  in  changing  references  in  the  TRD  and  master- 
slave  techniques.  It  results  in  reaveraging  phase  errors  from  the  remaining  references  in  the 
mutual  sync  technique.  As  a result  of  these  types  of  stresses  the  master-slave  technique  showed 
peak  frequency  errors  up  to  3.25  parts  in  10  The  mutual  sync  technique  showed  peak  frequency 
changes  of  from  10  to  20  times  that  of  master-slave  while  no  variation  in  nodal  frequency  was 
experienced  with  the  TRD  technique  due  to  these  stresses. 

The  response  of  the  TRD  node  to  a VCO  center  frequency  cha. .ge  was  due  to  a 
time  skew  in  the  components  that  make  up  the  total  time  reference  signal  with  respect  to  the 
master  at  nodes  further  down  the  timing  chain  than  where  the  stress  occurred.  This  time  skew  in 
error  components  was  a result  of  the  method  of  implementation  of  the  simulator  but  is  not  inherent 
to  the  TRD  technique.  This  response  does  not  die  out  in  going  further  out  the  chain  as  with  the 
master-slave  technique  because  with  TRD  each  succeeding  node  receives  the  same  error  due  to 
the  skewed  samples. 

The  master-slave  and  TRD  techniques  experienced  the  same  change  in  nodal 
frequency  when  switching  from  tracking  to  acquisition  mode.  However,  this  is  strictly  a function 
of  the  loop  parameters  and  phase  threshold  at  which  the  switch  occurs. 


r 
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Table  7. 3. 6. 4.  Summary  of  Generalized  Simulation  Results 


DISTURBANCE  PEAK  FREQUENCY  CHANGE  (M/f  x 1010) 

o 


NODES 

AWAY 

MUTUAL 
SYNC  ' 

MASTER- 

SLAVE 

TRD 

LINK  DROPOUT 
OF  NODE 
FAILURE 

1 

MAX.  - 65 
NOM.  - 20-30 

MAX.  3.25 
NOM  2 

0 

2 

MAX  - 30 

MAX  2 

0 

NOM  3-10 

NOM  1 

CHANGE  IN 

1 

10-33 

81 

7 

VCO  CENTER 

FREQUENCY 

OF  1 x 10'8 

2 

0-10 

10 

7 

I 


SWITCH  FROM 
TRACKING  TO 
ACQUISITION 
MODE 


SATELLITE 

LINK 

DISTURBANCE 


0 


1 MAX  7.  5 

2 2. 5 - 5.  5 


192 


192 


2(6A%A  ‘ 
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SECTION  8.0 

EVALUATION  OF  TIMING  TECHNIQUES 
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8.0  EVALUATION  OF  TIMING  TECHNIQUES 

In  the  paragraphs  that  comprise  this  section  each  of  the  four  timing  techniques, 
independent  clocks,  mutual  synchronization,  time  reference  distribution  and  master-slave  are 
analyzed  in  order  to  determine  the  degree  to  which  each  technique  provides  the  desirable  char- 
acteristics discussed  in  Section  4.0. 

8 . 1 Survivabi  lity 

Survivability  indicates  the  degree  to  which  the  timing  subsystem  continues  to 
perform  its  function  (to  provide  network  timing)  during  periods  of  stress.  Fallback  modes  of 
operation  shall  be  provided  so  that  the  timing  subsystem  itself  will  not  cause  signifiant  degrada- 
tion in  overall  DCS  network  capability  to  provide  dependable  digital  communications  during 
these  periods.  Survivability  is  important  because: 

1 . One  of  the  primary  goals  of  the  DCS  is  to  provide  dependable  digital 
communications  for  its  users  during  periods  of  network  stress. 

2.  The  network  timing  subsystem  must  be  operational  to  provide  acceptable 
digital  communication  during  such  periods. 

Network  synchronization  is  necessary  because  the  slip  rate  that  two  nodes  will  experience  in 
communicating  is  directly  proportional  to  the  frequency  offset  between  their  nodal  clocks.  If 
this  offset  increases  in  a stressed  environment  then  slip  rate  increases  which  leads  to  misaligned 
frames  in  multiplexers  and  switches  and  a need  to  resynchronize  cryptos.  Since  timing  subsystem 
performance  is  directly  tied  to  slip  rate,  we  will  measure  slip  rate  as  a function  of  level  of 
stress  to  indicate  the  survivability  of  each  timing  approach. 

There  are  two  parts  to  the  problem  of  minimizing  the  slip  rate  in  a digital  network: 

1 . Keep  the  nodal  clocks  of  all  communicating  node  pairs  synchronous  as  much 
as  possible. 

2.  Minimize  the  frequency  offset  between  the  nodal  clocks  of  asynchronous 

communicating  nodes  and  minimize  the  amount  of  time  they  must  communicate 
with  asynchronous  nodal  clocks. 
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To  put  these  comments  in  perspective  we  note  that  the  independent  clock  tech- 
nique uses  asynchronous  clocks  at  every  node.  The  slip  rate  due  to  buffer  overflows  is  strictly  a 
function  of  the  clock  accuracy  and  the  buffer  size  and  will  not  change  as  a function  of  stress 
level.  On  the  other  hand,  one  could  reduce  the  slip  rate  by  slaving  these  clocks  to  a common 
master  through  some  master-slave  timing  distribution  hierarchy.  This  would  make  all  nodal  clocks 
synchronous  and  slips  due  to  buffer  overflows  could  be  avoided  entirely  providing  there  were  no 
link  or  node  failures.  If  link  or  node  failures  occur,  then  some  nodes  may  lose  their  reference 
to  the  master.  Then  some  nodes  with  asynchronous  nodal  clocks  may  be  communicating  and 
could  experience  slips.  We  will  call  such  nodes  "asynchronous  communicating  nodes."  Such 
nodes  will  remain  asynchronous  until  they  once  again  become  locked  to  a common  reference. 

It  is  very  desirable  that  the  timing  approach  minimize  the  period  of  time  during  which  such 
nodes  are  asynchronous. 

From  these  comments  one  can  see  that  we  take  issue  with  the  common  misconcep- 
tion that  the  independent  clock  approach  offers  the  ultimate  in  survivability.  We  will  argue 
that  a disciplined  technique  with  a fixed  timing  distribution  network  has  better  survivability 
because  nodes  will  not  experience  slips  as  long  as  they  can  obtain  a reference,  and  when  a 
reference  is  not  available,  they  revert  to  a self-reference  approach  which  has  performance 
equivalent  to  an  independent  clock  approach.  However,  the  improvement  in  using  this  approach 
is  not  great  because  of  the  significant  probability  that  a node  will  lose  its  reference.  This  is 
the  reason  that  master-slave  approaches  have  been  criticized  for  having  poor  survivability. 
However,  there  is  a better  way  for  distributing  timing.  In  fact,  a dramatic  improvement  in 
survivability  can  be  obtained  by  using  an  adaptive  timing  distribution  network  approach  such  as 

92  205 

has  been  implemented  in  the  Canadian  Data  route  and  has  been  proposed  for  use  with  TRD. 

This  is  due  to  the  ability  of  each  node  connected  to  the  network  to  find  another  reference 

after  the  loss  of  a reference. 

We  have  performed  some  analyses  to  demonstrate  the  points  discussed  in  the  pre- 
ceding paragraph.  This  will  allow  suitable  comparisons  to  be  made  and  will  demonstrate  signifi- 
cant characteristics  of  each  approach. 
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The  independent  clock  approach  is  the  easiest  to  analyze.  All  communicating 
nodes  are  asynchronous.  The  time  between  slips  (or  buffer  overflows)  at  a node  can  be  calculated 


Ts  = B/<5 


(8.1-1) 


where  8 is  the  frequency  offset  (in  Hz)  between  the  two  nodal  clocks  and  B is  the  buffer  size 
(buffer  fill  may  change  by  up  to  ±B  bits  before  an  overflow  occurs).  The  performance  achieved 
with  this  approach  is  simply  a function  of  the  ratio  B/<5  . There  is  a direct  trade  between  buffer 
size  and  clock  accuracy.  One  may  reduce  the  clock  accuracy  by  a factor  of  K if  the  buffer  size 
is  increased  by  a factor  of  K,  and  the  value  of  T will  be  unchanged.  Note  also  that  the  per- 
formance is  independent  of  network  stress  level. 

The  performance  of  several  approaches  for  implementing  a disciplined  timing 
distribution  system  will  be  evaluated.  The  parameter  B/<5  is  considered  to  be  one  of  the 
fundamental  design  parameter  for  each  of  these  approaches.  In  this  case  the  clock  offset,  8 , 
refers  to  the  offset  between  any  two  nodal  clocks  that  will  occur  if  they  lose  their  references. 
Normally,  all  nodes  will  have  a common  reference.  Slips  will  not  occur  in  this  case.  The  only 
time  that  slips  will  occur  is  when  one  or  both  of  the  nodes  loses  its  reference  and  has  to  run 
"asynchronously."  A good  disciplined  approach  will  attempt  to  minimize  the  product  of  the 
amount  of  time  any  node  must  run  asynchronously  and  the  offset,  <J  , that  will  occur  during 
this  time . 

The  first  approach  to  be  considered  is  the  use  of  a fixed  timing  distribution  net- 
work. That  is,  even  though  each  node  may  have  several  links  over  which  it  is  communicating, 
it  will  only  have  one  link  from  which  it  may  derive  a reference.  If  that  reference  is  lost,  the 
node  must  run  asynchronously.  This  approach  will  have  the  poorest  performance  of  the  disci- 
plined approaches  in  terms  of  minimizing  the  amount  of  time  each  node  must  run  asynchronously. 
However,  it  performs  better  than  the  independent  clock  approach.  The  performance  will  be 
dependent  on  topology  of  the  network.  Because  of  this,  completely  general  results  cannot  be 
obtained.  However,  by  assuming  a specific  topology  that  can  be  analyzed,  one  can  obtain 
results  that  indicate  the  important  characteristics  of  this  approach.  The  topology  that  will  be 
used  is  the  "starred  polygon"  with  connectivity  4.  This  topology  for  an  8 node  network  is 


- 

shown  in  Figure  8.1  . Node  1 is  the  master,  and  timing  is  distributed  via  the  routes  indicated 
by  the  arrows.  Note  that  there  are  four  independent  timing  chains  that  distribute  timing 
(1-2-4;  1-3;  1-7-5;  1-8-6). 
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Figure  8.1.  Starred  Polygon  With  Connectivity  of  4 and  8 Nodes 

This  topology  was  chosen  mainly  for  convenience  in  analysis.  This  particular 
problem  is  very  tedious  to  analyze  for  any  general  topology.  However,  results  can  be  obtained 
for  regularly  structured  topologies  such  as  that  shown  in  Figure  8.1  . Thus,  the  results  will  serve 
mainly  to  demonstrate  important  characteristics  of  the  approaches  being  analyzed  rather  than 
provide  highly  accurate  comparisons  which  will  be  valid  for  general  network  topologies.  Such 
results  cannot  be  obtained. 

The  details  of  the  analysis  are  presented  in  Appendix  B.  The  analysis  will  consider 
only  link  failures  for  purposes  of  simplicity.  Addition  of  node  failures  to  the  model  would  odd 
significant  complexity  to  the  analysis  effort  without  significantly  improving  the  results.  The 
average  fraction  of  asynchronous  communicating  node  pairs  was  calculated  and  is  given  by  Equation 
(B.3).  Asymptotically,  for  small  p,  T)  Fac  « kp,  where  p is  the  probability  of  link  outage. 

Thus,  as  one  might  expect  since  single  link  outages  can  cause  many  nodes  to  lose  their  timing 

references  this  term  dominates  Fac.  The  worst  case  is  for  large  p -1  for  which  Fac  is  almost 

0.5.  This  is  shown  in  Figure  B.2.  These  results  were  related  to  the  time  between  slips  by 
observing  that  slips  only  occur  on  that  fraction  of  the  nodes  that  are  asynchronous.  Then 
T$=  B/<5  Fac  . (8.1-2) 

Since  the  worst-case  Fac  a 0.5  then  T^  « 2B/<J  . Thus,  this  approach  has  an  average  time 
between  slips  of  about  twice  that  of  an  independent  clock  approach  with  the  same  value  of  B/S. 
However,  at  small  values  of  p the  time  between  slips  improves  dramatically  as  shown  in  Figure 

B.3. 
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Next,  we  analyzed  a system  with  a primary  reference  and  a fixed  alternate 

reference.  The  same  network  topology  was  used,  and  the  primary  reference  was  the  same  as  is 

shown  in  the  figure.  It  was  assumed  that  each  node  took  its  alternate  reference  from  the  node 

adjacent  to  it  looking  back  toward  the  master.  Again,  Fac  was  calculated  and  is  given  by 

2 

(B.5).  Asymptotically,  for  small  p,  we  find  that  Fac  « kp  . This  is  consistent  with  our  approach 
of  providing  two  available  references  per  node.  Then  two  link  outages  can  cause  a node  to  lose 
its  reference  to  the  master  and  become  asynchronous  with  respect  to  the  rest  of  the  network. 

The  worst-case  occurs  for  p « 0.5  for  which  Fac  « 0.11  for  a 20  node  network.  At  this  value 
of  p , T ar  9B/S  which  is  a considerable  improvement  but  still  does  not  offer  dramatically 
higher  survivability  than  the  independent  clock  approach.  These  results  are  also  shown  in 
Figures  B *2  and  B .3 . 

The  final  approach  considered  was  a disciplined  system  with  the  capability  of 

adaptively  reconfiguring  the  timing  distribution  network  as  link  outages  occur.  This  technique 

go 

has  been  implemented  in  the  Dataroute  network'^  and  has  been  proposed  for  use  with  Time 
205 

Reference  Distribution.  This  approach  has  the  properties  that: 

1 . All  communicating  nodes  are  synchronous  in  the  steady-state  (after  a 
sufficiently  long  period  following  the  last  failure). 

2.  The  only  time  a nodal  clock  will  act  as  a self-reference  is  while  the  timing 
distribution  network  is  being  reconfigured  (which  is  a relatively  short  period 
of  time) . 

This  approach  has  a high  degree  of  survivability.  The  system  can  be  designed  so  that  no  slips 
occur  in  the  steady-state.  The  only  time  slips  can  occur  is  during  the  transients  associated 
with  reconfiguring  the  network.  Obviously,  one  of  these  transients  occurs  every  time  an  outage 
occurs,  and  any  single  outage  can  cause  the  network  to  be  reconfigured  in  such  a manner  that 
several  nodes  may  have  to  act  as  a self-reference  during  this  process.  Performance  in  a stress 
environment  is  influenced  by  both  the  time  required  to  reconfigure  the  timing  distribution 
network  and  the  length  of  time  a node  can  act  as  a self-reference  without  experiencing  slips 
(which  is  a function  of  the  buffer  size  and  frequency  offset). 


There  is  a trade-off  here  between  time  required  for  reconfiguration,  buffer  size, 
and  the  fractional  frequency  offset  between  the  nodal  clock  and  the  network  frequency.  For 
example,  a cesium  clock  could  operate  without  reference  for  24  hours  while  accumulating 
only  about  1 ps  of  phase  error  relative  to  the  network  master.  In  actual  practice,  though  it 
should  be  no  problem  to  accomplish  network  reconfiguration  is  10  minutes.  In  this  case  even 
the  phase  errors  accumulated  with  quartz  clocks  will  be  rather  small. 

The  only  way  that  this  approach  can  encounter  a problem  is  if  it  must  spend  all 
of  its  time  attempting  to  find  a suitable  reference  without  ever  obtaining  a reference  for  a 
period  long  enough  to  allow  it  to  resynchronize.  In  order  for  this  to  happen  the  reference  must 
fail  soon  after  the  node  tries  to  lock  to  it.  A rather  crude,  but  conservative,  analysis  was 
carried  out  to  relate  T to  link  failure  rate  as  a function  of  certain  design  parameters.  The  results 
are  shown  in  Figure  B .4 . This  approach  is  many  orders  of  magnitude  better  than  the  independent 
clock  approach  with  the  same  value  of  B/S  even  at  very  high  failure  rates.  In  fact,  the  larger 
the  value  of  B/S,  the  greater  the  advantage  of  using  this  adaptive  approach.  For  example,  for 
B/S  = 24  hours  this  approach  is  7 orders  of  magnitude  better  than  the  independent  clock  approach 
even  with  Fd  = 10  (i.e.,  10  link  failures  per  day  per  link).  One  would  expect  at  most  fust  a 
few  failures  per  day  per  link  even  under  severe  stress  because  a high  level  of  stress  would  imply 
greater  difficulty  in  making  repairs.  Certainly,  there  would  be  many  failures  caused  by  physical 
damage  which  could  take  days  or  longer  to  repair.  Thus,  the  nature  of  most  failures  is  not  such 
that  a link  may  fail,  be  repaired  in  a few  minutes,  then  fail  again  a short  time  later  followed 
by  a quick  repair,  etc.  Certainly  this  type  of  failure  mechanism  which  could  lead  to  a higher 
value  of  F^  poses  the  only  threat  to  a disciplined  timing  subsystem  with  adaptive  reorganization. 
The  reason  for  this  is  that  links  which  behave  in  this  way  cause  the  network  to  have  to  reorganize 
too  often.  Such  behavior  could  be  caused  by  a link  which  was  frequently  jammed  for  short 
periods  of  time.  Thus,  an  enemy  could  try  to  disrupt  the  synchronization  subsystem  by  frequently 
jamming  a number  of  links.  However,  this  tactic  can  be  effectively  countered  by  not  allowing 
any  node  to  use  a link  as  a reference  which  has  failed  more  than  a few  times  a day. 
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These  results  indicate  that  the  performance  of  master-slave  timing  subsystem  with 
adaptive  reorganization  is  greatly  superior  than  that  of  an  independent  clock  timing  subsystem 
at  virtually  all  stress  levels  even  using  much  smaller  values  of  B/ 8 . One  might  add  that  pre- 
cisely the  same  comments  could  be  made  about  the  time  reference  distribution  approach  because 
it  also  uses  the  adaptive  reorganization  technique.  This  is  the  key  to  achieving  a high  level 
of  performance  under  stress.  In  fact,  the  TRD  approach  should  perform  slightly  better  than  the 
master-slave  approach  because  of  its  ability  to  remove  the  effects  of  path  delay  variations  on 
nodal  frequencies.  This  will  allow  it  to  operate  at  a slightly  better  offset,  8 , from  the  network 
master. 

The  mutual  synchronization  approach  also  achieves  a high  degree  of  survivability. 

In  fact,  one  might  even  think  that  it  should  perform  better  under  stress  than  the  adaptive  reorga- 
nization approach  since,  if  a node  has  at  least  one  operational  link  it  has  a reference.  Thus,  in 
a pure  mutual  sync  network  nodes  never  communicate  asynchronously.  In  actuality,  though,  the 
master-slave  and  TRD  systems  with  adaptive  reorganization  can  be  designed  so  that  even  under 
severe  stress,  the  probability  that  node  pairs  have  to  communicate  asynchronously  for  long  enough 
to  result  in  slips  (or  buffer  overflows)  is  extremely  small.  Thus,  the  mutual  sync  approach  would 
offer  no  measurable  advantage.  In  fact,  it  is  likely  that  there  will  be  more  variation  in  nodal 
clock  frequencies  (or  more  error  propagation)  with  the  mutual  sync  approach  under  stress  which 
is  an  undesirable  characteristic. 


8.2 


Error  Propagation 


One  of  the  principal  objectives  of  a network  synchronization  scheme  is  to  maintain 
very  stable  nodal  frequencies  at  each  node  in  the  network.  Thus,  it  is  highly  desirable  to  prevent 
perturbations  in  the  frequency  of  one  nodal  clock  from  propagating  and  influencing  the  frequency 
of  nodal  clocks  in  other  parts  of  the  network.  By  minimizing  such  perturbations  of  nodal  clocks, 
one  maximizes  the  stability  of  each  nodal  clock  and  minimizes  the  probability  of  a bit  slip. 

The  degree  of  error  propagation  for  several  types  of  disturbances  will  be  evaluated 
for  each  of  the  synchronous  timing  techniques,  master-slave,  mutual  sync,  and  time  reference 
distribution.  Four  types  of  disturbances  will  be  evaluated.  These  are  a link  dropout,  a step 
change  in  nominal  VCO  frequency,  a step  change  in  path  delay,  and  sinusoidal  path  delay 
variations.  The  disturbances  will  be  applied  at  one  point  in  the  network  and  the  effect  on  other 
nodal  frequencies  will  be  measured  as  a function  of  distance  (in  number  of  nodes)  from  the  dis- 
turbance. This  will  be  evaluated  entirely  by  simulation.  The  results  of  the  simulations  will  be 
summarized  in  tabular  form  for  each  type  of  disturbance.  In  addition,  plots  of  nodal  frequencies 
as  a function  of  time  after  a link  dropout  will  be  shown  for  the  three  timing  techniques. 


Two  network  configurations  will  be  used  in  these  evaluations.  The  network  shown 
in  Figure  8.2-1  will  be  used  to  evaluate  master-slave  and  TRD.  Node  1 is  the  master,  and  the 

Maste  r 


timing  distribution  chains  are  1-3-5-7-22-23  and  1-4-6-8-20-24.  The  disturbances  will  occur 
on  the  second  of  these  two  chains  and  will  propagate  down  the  chain.  The  first  chain  will  remain 
undisturbed,  and  comparisons  between  the  two  can  be  made.  One  could  also  use  this  network  to 
test  mutual  sync  by  placing  a disturbance  at  the  left  end  and  determining  the  degree  of 
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propagation.  However,  this  configuration  is  probably  not  representative  of  how  errors  will 
typically  propagate  in  a mutual  sync  network.  For  this  reason,  we  have  chosen  to  use  the  more 
fully  connected  network  of  Figure  8.2-2.  Propagation  of  disturbances  to  distant  nodes  can  occur 
over  many  parallel  paths.  In  general,  there  will  be  multiple  nodes  at  distance  one,  two,  three, 
etc.  The  results  presented  will  display  the  average  effect  on  all  nodes  at  distance  one,  two, 
three,  etc. 


Dropout 

Offset 


Figure  8.2-2.  Mutual  Sync  Error  Propagation  Experiment 

Two  simulations  were  performed  to  compare  perturbations  due  to  path  delay  varia- 
tions. In  the  first,  a step  change  in  path  delay  with  a magnitude  of  0.65  ps  was  introduced  on 
one  link  (Links  1-4  in  the  master-slave  and  TRD  simulations  and  Links  23-24  in  the  mutual  sync 
simulation).  The  peak  fractional  change  in  nodal  frequency  as  a function  of  distance  from  the 
disturbance  for  these  simulations  is  shown  in  Table  8.2-1.  The  results  indicate  that  mutual  sync 
is  relatively  less  affected  by  the  step  change  than  master-slave.  The  reason  is  that  the  link  with 
delay  variation  is  only  one  of  several  being  averaged  to  obtain  a reference  at  the  nodes  nearest 
the  disturbance.  However,  there  are  two  undesirable  properties  of  mutual  sync  which  are  not 
reflected  in  Table  8.2-1.  First,  the  disturbance  causes  a slight  permanent  change  in  system 
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Table  8.2-1.  Error  Propagation  Due  to  a 0.65  ps  Step  Change 

in  Path  Delay 

Distance  Peak  Af/fo 


(Nodes) 

Master-Slave 

Mutual  Sync 

TRD 

1 

0 

1 

o 

X 

CO 

0 

1 

o 

- 

2 

o 

1 

o 

3 X 10-1 1 

- 

3 

„ ,„-ll 

6x10 

2.5  x lO"11 

- 

4 

1 

o 

X 

2.5  x lO"’1 

_ 

frequency.  Second,  the  disturbance  will  propagate  to  all  nodes  in  the  network  in  a mutual  sync 
system,  but  the  disturbance  will  propagate  only  to  nodes  lower  in  the  timing  distribution  chain  in 
a master-slave  system  thereby  affecting  only  a small  fraction  of  all  the  nodes  in  the  network.  The 
simulation  of  TRD  displayed  no  measurabie  effect  on  nodal  frequencies  due  to  the  step  change  in 
path  delay.  This  indicates  the  advantage  of  using  a double-ended  system  to  eliminate  the  effect 
of  path  delay  variations. 

Rapid  path  delay  variations  are  effectively  attenuated  by  all  three  techniques,  and 
there  is  very  little  effect  on  nodal  frequencies.  However,  very  slow  path  delay  variations  will 
perturb  the  nodal  frequencies.  A typical  case  is  that  of  variations  with  a period  of  1 day.  This 
case  was  simulated  with  the  variation  placed  on  all  links  in  the  network.  The  magnitude  of  the 
variation  was  0.65  ps  (peak-fo-peak)  on  each  link.  The  results  of  the  simulations  are  shown  in 
Table  8.2-2. 

Table  8.2-2.  Error  Propagation  Due  to  a Sinusoidal  Path  Delay 
Variation  on  All  Links  With  a Magnitude  of  0.65  ps 
Peak-to-Peak  and  a Period  of  1 Day 

Distance  Peak  Af/fo 


(Nodes) 

Master-Slave 

Mutual  Sync 

TRD 

1 

2.5  x 10"11 

1 .4  x 10"10 

- 

2 

5 x lO"11 

o 

o 

X 

- 

3 

7 x lO"11 

1.4  X 10"10 

- 

4 

1 

o 

X 

cs 

X 

o 

1 

o 

_ 
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Neither  master-slave  nor  mutual  sync  successfully  attenuates  the  effect  of  the  path  delay  varia- 
tions. The  distance  shown  for  master-slave  refers  to  the  distance  from  the  master.  The  node  that 
is  distance  one  away  receives  the  effect  of  only  one  link  variation,  the  node  at  distance  two 
receives  the  effect  of  two  link  variations,  etc.  Thus,  the  perturbation  of  nodal  frequencies  due 
to  path  delay  variations  grows  as  a function  of  distance  from  the  master.  Of  course,  this  result 
is  to  be  expected,  and  typically  the  requirements  for  stable  nodal  frequencies  are  less  severe  at 
the  tails  of  the  network.  In  contrast,  we  note  that  all  nodes  in  a mutual  sync  system  are  perturbed 
to  the  same  degree.  The  amount  of  the  perturbation  at  all  nodes  in  a mutual  sync  network  is  about 
twice  that  experienced  at  the  third  level  of  the  master-slave  network.  Thus,  one  would  expect 
that  the  perturbation  in  a master-slave  network  would  not  be  worse  than  that  in  this  mutual  sync 
network  until  the  sixth  or  seventh  level  ot  the  network.  The  mutual  sync  network  of  Figure  8.2-2 
could  have  been  configured  as  a master-slave  network  with  all  timing  chains  of  length  three  or 
shorter,  and  thus  master-slave  would  perform  considerably  better  than  mutual  sync  on  it.  One 
would  expect  similar  results  for  larger  networks.  The  effect  of  path  delay  variations  on  a mutual 
sync  network  will  grow  as  a function  of  network  size  since  these  perturbations  propagate  through- 
out the  network . 

We  note  that  once  again  TRD  is  unaffected  by  path  delay  variations.  The  magnitude 
of  the  fractional  frequency  variations  in  master-siave  and  mutual  sync  networks  due  to  path  delay 
variations  is  several  parts  in  10^.  Because  of  this,  one  cannot  take  advantage  of  the  inherent 
stability  of  atomic  clocks  at  nodes  other  than  a master  node  using  these  techniques.  However, 
by  using  the  double-ended  technique  of  TRD,  one  can  use  more  stable  clocks  and  improve  the 
stability  of  nodal  frequencies  by  an  order  of  magnitude  to  a few  parts  in  10^. 

The  third  disturbance  simulated  was  a link  outage.  Link  23-24  was  removed  in  the 
mutual  sync  network.  For  master-slave  and  TRD  Link  1-4  was  removed  and  Node  4 then  slaved 
to  Node  3.  The  results  of  the  simulation  are  summarized  in  Table  8.2-3.  The  output  from  the 
simulation  runs  is  shown  in  Figures  8.2-3,  8.2-4,  and  8.2-5.  Note  that  the  scale  for  fractional 
frequency  offset  is  different  for  mutual  sync  than  for  master-slave  and  TRD.  Several  observations 
can  be  made.  First,  mutual  sync  is  much  more  severely  affected  by  a link  dropout  than  the  other 
approaches.  The  reason  is  that  the  VCO  at  each  node  must  be  held  to  the  network  frequency  by 
a nonzero  steady-state  phase  error.  If  a node  is  averaging  the  phase  error  on  n links  to  provide 
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Table  8.2-3.  Error  Propagation  Due  to  a Link  Outage 


Distance 

Peak  A f/fo 

(Nodes) 

Master-Slave 

Mutual  Sync 

TRD 

1 

1.7  x 10"’° 

4.5  x 10"9 

6.5  x 10' 

11 

2 

6.5  x 10-11 

0 

1 

o 

X 

CO 

6.5  x 10" 

11 

3 

4.5  x lO"11 

X 

o 

1 

o 

6.5  x 10 

11 

4 

4 x lO"11 

4 X 10-10 

6.5  x 10 

11 

an  input  for  the  loop  filter,  then  it  must  average  the  other  n-1  finks  when  one  of  the  references 

is  lost.  At  that  instant,  the  average  of  the  phase  errors  on  the  n-1  links  can  be  considerably 

different  from  the  average  of  the  phase  errors  on  the  original  n links.  Thus,  the  loop  can  be 

hit  with  a large  phase  step.  The  magnitude  of  this  effect  can  be  quite  variable,  and  one  principal 

contributor  is  the  offset  between  the  sytem  frequency  and  the  nominal  VCO  center  frequency  at  a 

node  which  loses  a link  (since  bigger  offsets  result  in  bigger  phase  errors).  In  this  simulation  the 

_8 

fractional  offsets  at  Nodes  23  and  24  (Curves  1 and  2 in  Figure  8.2-3)  were  approvimately  10 
which  is  normal  for  good  quartz  crystal  VCO's.  As  we  discussed  in  Paragraph  3.2.4  one  could 
possibly  reduce  the  magnitude  of  this  transient  by  not  allowing  such  a large  step  change  in  phase 
error  be  applied  to  the  loop  filter  but  making  it  occur  more  slowly.  Even  if  this  were  successful, 
one  would  still  have  permanent  changes  in  the  steady-state  system  frequency  due  to  link  dropouts. 
Such  a change  was  not  observed  in  this  simulation  because  the  scale  chosen  for  the  printout  was 
not  adequate,  but  in  previous  simulations  the  steady-state  frequency  has  changed  by  an  amount 
about  an  order  of  magnitude  less  than  the  peak  frequency  change. 

We  note  that  the  perturbation  on  master-slave  is  about  a factor  of  20  less  than  that 
of  mutual  sync.  The  reason  there  is  any  perturbation  at  all  is  that  the  references  arriving  on 
Links  1-4  and  3-4  are  not  in  phase  with  each  other.  Thus,  the  loop  must  recover  from  a phase 
step  after  the  switchover  to  the  alternate  reference. 

During  the  switchover  to  an  alternate  reference  with  TRD,  the  fact  that  the  refer- 
ences arriving  on  Links  1-4  and  3-4  are  not  in  phase  will  not  cause  a transient  (because  TRD  is  a 
double-ended  system).  However,  if  there  are  constant  link  asymmetries  on  the  two  links  that  are 
different,  then  the  effect  will  be  the  same  as  if  the  loop  were  perturbed  by  a phase  step.  In  this 
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Figure  8.2-3.  Nodal  Frequencies  of  Master-Slave  After  a Link  Dropout 


Figure  8.2-4.  Nodal  Frequencies  of  Mutual  Sync  After  a Link  Dropout 


. — . 





1 * • • • 

• • • • j 

• 

1 

1 

1 . , 

, 

1 ' 4 ’ 

; 

i 

...  i 

' 

...... 

: 

i 

i 

i 

L ■ _iil 

■ 

i ■ ■ 
.... 

t 

■ 

■ 1 

• 

4— ■ - ■ ■*■■' 

' ' 1 

1 

. . . . 

simulation,  the  link  asymmetry  was  300  ns  on  link  1-3  and  0 on  link  1-4.  Thus,  the  difference 
in  link  asymmetries  is  300  ns.  Then  the  loop  responds  to  a phase  step  of  half  this  amount  or  150 
ns.  The  magnitude  of  this  asymmetry  term  is  probably  rather  conservative.  The  peak  frequency 
change  on  TRD  is  about  half  that  of  the  master-slave  approach  and  about  a factor  of  40  less  than 
mutual  sync.  If  the  asymmetry  term  was  different,  the  peak  frequency  change  would  scale 
accordingly.  This  simulation  run  assumed  the  quartz  crystal  VCO  loop  parameters.  Another  run 
using  the  cesium  clock  loop  parameters  improved  the  TRD  results  by  another  factor  of  10. 

The  final  simulation  was  a step  in  nominal  VCO  center  frequency  equal  to  a frac- 
-8 

tional  frequency  change  of  10  . The  frequency  steps  occurred  at  node  4 in  the  master-slave 

and  TRD  networks  and  at  node  23  in  the  mutual  sync  network.  Of  course,  this  results  in  a 
severe  frequency  transient  at  the  node  at  which  the  frequency  step  occurs.  The  three  techniques 
were  compared  on  the  basis  of  how  severely  the  adjacent  nodes  were  affected.  The  results  are 
shown  in  Table  8.2-4, 


Table  8.2-4.  Error  Propagation  Due  to  a Fractional  Frequency  Step 
-8 

of  10  in  Nominal  VCO  Center  Frequency 
Distance  Peak  Nf/fo 


(Nodes) 

Master-Slave 

Mutual  Sync 

TRD 

1 

o 

1 

o 

X 

00 

1.4  x Kf9 

00 

X 

o 

2 

6.5  X 10-11 

5xl0"10 

i 

o 

X 

00 

3 

1 .3  x 10-11 

5 x 10“10 

00 

X 

o 

1 

4 

1 .3  x 10"11 

5 x 10~10 

00 

X 

o 

1 

We  note  that  master-slave  is  better  than  mutual  sync  by  about  a factor  of  two. 

The  TRD  simulation  used  the  wider  band  loop  used  with  quartz  crystal  clocks.  It 
had  the  same  peak  Af  as  master-slave  at  distance  one,  but  its  comparison  suffered  at  greater 
distances  because  the  transient  was  not  filtered  out.  All  nodes  further  down  the  timing  chain 
behave  virtually  the  same  as  the  node  nearest  the  disturbance.  Normally,  a perturbation  in  a 
nodal  clock  frequency  in  the  TRD  technique  should  not  affect  the  frequencies  at  clocks  further 
down  in  the  timing  chain  since  all  phase  errors  are  referenced  to  the  master.  However,  in  order 
to  do  this  the  quantities  used  to  compute  the  phase  error,  AT,  , relative  to  the  master 
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(i.e.,  AT  „ , AT  , and  AT.)  must  all  be  measured  at  the  same  time.  In  our  simula- 
tion  we  did  not  do  this.  The  quantities  AT  and  \T  were  computed  1 minute  earlier  (in 
simulated  real  time)  than  the  value  of  AT^  to  which  they  were  added.  This  introduced  the  meas- 
urement error  which  allowed  the  effects  of  the  frequency  step  to  propagate.  Had  these  measure- 
ments been  perfectly  synchronized  no  effect  would  have  been  observed.  Obviously,  one  could 
do  a much  better  job  of  synchronizing  these  measurements  than  we  did,  and  the  benefit  would  be 
proportionately  better  performance,  i.e.,  if  they  could  have  been  synchronized  to  within  6 
seconds,  the  error  propagation  would  have  been  reduced  by  a factor  of  10.  Simulations  of  the 
cesium  clock  loop  parameters  at  all  nodes  following  the  node  with  the  frequency  offset  (node  4) 
indicate  that  the  transient  at  other  nodes  is  reduced  by  another  order  of  magnitude  due  to  the 

smaller  value  of  cj  • In  addition,  if  the  clock  at  node  4 had  been  a cesium  clock  with  the 
n 

cesium  clock  loop  parameters,  then  a frequency  step  as  large  as  was  experienced  (one  part  in 

g 

10  ) could  have  never  happened.  Any  such  frequency  step  would  have  been  about  one  part  in 
101  . In  th  is  case  the  transients  would  have  been  reduced  by  an  additional  order  of  magnitude. 
Actually,  the  comparisons  between  master-slave  and  TRD  are  so  close  on  this  disturbance  that  we 
would  have  to  say  they  were  even.  However,  the  performance  of  the  TRD  technique  can  be 
considerably  improved  as  discussed  above. 

In  summary,  we  have  tried  to  obtain  an  overall  ranking  of  the  three  approaches  on 
the  basis  of  the  four  types  of  disturbances.  Fortunately,  the  results  were  virtually  the  same  on 
each  evaluation.  A summary  of  the  rankings  for  each  type  of  disturbance  and  the  overall  rankings 
are  shown  in  Table  8.2-5. 

Table  8.2-5.  Ranking  of  Timing  Techniques  on  the  Basis  of  Error 
Propagation  for  Four  Types  of  Disturbances 

Disturbance  Ranki 


ing 


Path  Delay  Step 

Sinusoidal  Path  Delay  Variation 
Link  Dropout 
Frequency  Offset 
Overall  Ranking 


Master-Slave 

3 

2 

2 

1 

2 


Mutual  Sync 
2 
3 
3 
3 
3 


TRD 


1 (tie) 
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Mutual  sync  was  third  in  the  comparisons.  Not  only  did  a disturbance  typically 
affect  other  nodes  at  a given  distance  to  a greater  degree  than  the  other  techniques,  but  if 
affected  many  nodes  since  disturbances  in  TRD  and  master-slave  would  affect  only  those  nodes 
in  the  same  timing  chain.  On  the  other  hand,  time  reference  distribution  offers  a clear  advantage 
over  the  other  two  techniques  in  maintaining  highly  stable  nodal  frequencies  under  normal  operating 
and  stress  conditions.  The  reason,  of  course,  is  due  to  the  double-ended  nature  of  the  technique. 
The  resulting  stability  of  nodal  frequencies  is  a few  parts  in  10^  which  is  very  close  to  that  of 
cesium  standards.  Thus,  one  can  effectively  utilize  atomic  standards  in  the  network  at  those 
nodes  where  highly  stable  nodal  frequencies  are  required  with  the  TRD  approach.  The  cheaper 
quartz  crystal  standards  can  be  used  at  other  nodes  with  slightly  degraded  performance,  but  even 
at  these  nodes  the  nodal  frequencies  will  be  more  stable  than  if  master-slave  or  mutual  sync  were 
used . 
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The  compatibility  of  the  DCS  timing  subsystem  with  other  global  timing  sources  is 
a desirable  characteristic  because  it  can  conceivably  result  in  an  increase  in  the  availability 
of  the  timing  function  for  the  DCS.  Such  an  increase  in  availability  results  when  the  external 
timing  source  provides  timing  for  a node  after  a failure  that  prevents  the  node  from  attaining 
timing  internal  to  the  DCS. 

In  order  to  compare  the  alternative  timing  techniques  with  respect  to  their  compat- 
ibility to  external  timing  subsystems,  two  points  must  be  examined:  First,  do  any  of  the  timing 
techniques  facilitate  the  hardware  implementation  necessary  to  provide  this  compatibility? 

Second,  do  any  of  the  techniques  result  In  a more  effective  transfer  from  primary  to  backup 
timing  sources?  By  more  effective,  one  means;  do  the  buffers  at  the  node  switching  to  the 
backup  timing  source  and  for  that  matter,  all  buffers  in  the  network,  fill  or  empty  more  slowly 
for  a timing  subsystem  based  on  one  specific  technique? 

A third  point  concerning  compatibility  with  other  timing  sources  is  whether  or  not 
it  is  a cost-effective  method  for  providing  additional  redundancy  for  the  timing  function.  Since 
this  point  impacts  the  overall  significance  of  compatibility  rather  than  the  comparison  of  levels 
of  compatibility  between  techniques,  it  has  been  addressed  in  Section  10.0  dealing  with  the 
ranking  of  desirable  characteristics. 

Timing  subsystems  for  the  DCS  based  on  independent  clocks,  mutual  synchronization, 
master-slave  or  time  reference  distribution  can  all  be  designed  to  utilize  an  external  timing 
source.  Timing  sources  such  as  LORAN  C,  OMEGA  or  GPS  can  be  used  as  the  backup  source. 

In  comparing  the  requisite  hardware  necessary  to  incorporate  an  external  reference  as  a backup 
timing  source,  there  is  little  difference  between  any  of  the  techniques.  This  is  true  especially 
from  a cost  point  of  view  where  the  dominant  cost  by  far  is  the  receiver  necessary  for  the  backup 
timing  source.  The  functional  implementation  for  the  master-slave  technique  is  merely  to  input 
the  timing  signal  derived  from  the  external  receiver  to  the  switch  that  selects  the  appropriate 
reference  that  disciplines  the  local  clock.  In  the  mutual  synchronization  technique,  the  func- 
tional implementation  merely  requires  the  addition  of  a switch  to  which  both  the  external  and 
internal  timing  signals  are  applied.  The  selection  can  then  be  made  as  to  which  timing  signal 
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to  utilize  in  disciplining  the  local  oscillator.  In  the  independent  clock  approach,  a switch 
would  be  included  to  inject  the  timing  signal  derived  from  the  external  receiver  to  the 
appropriate  distribution  amplifiers  that  would  under  ordinary  operating  conditions  accept  a 
signal  from  the  local  clock.  In  the  time  reference  distribution  approach,  the  local  oscillator 
that  had  been  disciplined  via  a time  transfer  technique  would  now  be  disciplined  via  the 
external  timing  signal. 

In  summary,  the  hardware  implementations  appear  only  marginally  different 
and  therefore,  all  four  techniques  would  rate  equal  in  this  respect. 

The  second  point  to  be  explored  is  the  transient  with  respect  to  buffer  fill  that 
results  when  the  backup  timing  source  is  used  at  one  or  a number  of  nodes.  Buffers  begin  to 
fill  or  empty  relative  to  their  previous  fill  when  the  alternate  timing  source  is  introduced  at  a 
node  becduse  the  frequency  of  the  new  source  will  not  be  precisely  the  same  as  the  frequency  of 
the  original  source.  In  comparing  the  different  techniques,  the  major  question  to  be  raised  is 
the  size  of  the  frequency  step  between  primary  and  backup  timing  sources.  The  bigger  the 
frequency  step,  the  faster  the  buffers  will  fill  or  empty  and  the  greater  the  probability  of  bit 
slip.  As  is  mentioned  in  several  places  in  this  report,  it  appears  likely  that  a time  reference 
distribution  approach,  if  implemented,  would  be  ultimately  referenced  to  a world-wide  time 
standard  such  as  UTC.  Similarly,  potential  backup  timing  sources  such  as  LORAN-C  are  also 
referenced  to  UTC.  Thus,  it  can  be  anticipated  that  a switch  to  a backup  timing  source  in  a 
subsystem  based  on  TRD  will  result  in  a smaller  frequency  step  than  any  of  the  other  techniques 
since  both  primary  and  backup  systems  could  very  well  be  referenced  to  the  same  ultimate  source. 
The  second  point  to  be  considered  is  the  degree  to  which  the  frequency  step  at  the  single  node 
affects  the  frequency  at  the  other  nodes  of  the  network.  This  point  has  been  examined  in 
detail  in  the  evaluation  of  other  desirable  characteristics  (see  error  propagation).  Again, 
it  appears  from  our  simulation  results  that  TRD  provides  superior  insulation  for  the  remainder 
of  the  network  than  do  the  other  disciplined  clock  approaches.  Naturally,  there  is  infinite 
isolation  in  the  independent  clock  approach. 

In  summary,  TRD  must  be  given  the  highest  ranking  of  the  alternative  techniques 
for  compatibility  based  on  the  likelihood  of  a minimum  frequency  step  when  a backup  timing 
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source  is  employed.  The  remaining  alternatives  ranked  from  highest  to  lowest  are  independent 
clocks,  master-slave  and  mutual  synchronization  with  these  rankings  based  on  the  level  of 
insulation  of  the  frequency  step  in  the  remainder  of  the  network. 


8.4  Precise  Time  Availability 

Of  the  four  techniques  being  considered,  only  time  reference  distribution  has 
the  ability  to  provide  precise  time.  That  is,  TRD  has  the  ability  of  maintaining  time-of-day 
synchronization  between  each  of  the  network  clocks  and  the  internationally  accepted  time 
scale.  Coordinated  Universal  Time  (UTC).  As  is  noted  in  other  sections  of  this  report,  the 
availability  of  precise  time  is  not  a prerequisite  for  network  synchronization;  however,  benefits 
do  accrue  if  such  a capability  is  provided. 

Since  timing  subsystems  are  based  on  the  independent  clock  technique,  master- 
slave  technique  and  mutual  synchronization  technique  do  not  provide  precise  time,  and  a 
timing  subsystem  based  on  the  time  reference  distribution  technique  does  provide  precise  time, 
a comparison  of  the  techniques  for  this  particular  characteristic  is  unnecessary.  Rather,  a brief 
explanation  of  why  TRD  does  provide  precise  time  and  why  the  other  techniques  do  not  provide 
precise  time  appears  appropriate. 

Time  reference  distribution  can  provide  precise  time  at  each  nodal  clock  because 
the  timing  signal  that  disciplines  the  nodal  clock  is  precisely  the  timing  signal  that  is  generated 
at  the  node  to  which  it  is  slaved.  That  is,  the  disciplining  signal  has  not  been  corrupted  by 
the  path  delay  variations  inherent  in  the  link  connecting  the  two  nodes.  By  correcting  the 
slaved  clock  so  that  the  phase  error  between  the  disciplined  and  disciplining  signals  is  driven 
to  zero,  and  by  taking  as  an  ultimate  reference  a source  of  UTC,  precise  time  is  then  made 
available  at  each  node  in  the  network. 


Independent  clocks  by  definition  cannot  supply  precise  time  since  the  clocks  are 
not  disciplined.  Even  the  most  accurate  clocks  have  finite  frequency  differences  and  such 
frequency  differences  necessarily  result  in  an  accumulation  of  phase  difference,  thus  precluding 
the  availability  of  precise  time. 


Mutual  synchronization,  similarly,  by  definition  cannot  provide  precise  time. 
Even  if  a node  within  the  network  was  a source  of  UTC,  this  is  not  to  say  there  is  a rationale 
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for  including  UTC  in  a mutual  system.  The  simple  fact  that  each  nodal  frequency  is  determined 
by  an  average  of  the  frequencies  received  at  that  node  dictates  that  the  system  frequency 
will  not  be  the  same  as  the  frequency  provided  by  the  UTC  referenced  node.  Thus,  precise 
time  cannot  be  available  at  the  nodes  of  the  network. 

Master-slave  has  the  hierarchial  structure  necessary  for  the  distribution  of  time 
and  could  also  be  ultimately  referenced  to  a source  of  UTC.  However,  the  master-slave 
system  is  appropriate  for  the  distribution  of  time  since  it  cannot  completely  remove  the  path 
delay  variations  on  the  link  connecting  master  and  slave  nodes.  Unlike  the  TRD  technique,  the 
clock  at  the  slaved  node  in  the  master-slave  technique  is  disciplined  by  the  timing  signal 
received  at  that  node.  Although  appropriate  filtering  will  remove  much  of  the  path  delay 
variations,  over  a long  period  of  time,  phase  error  between  clocks  can  accumulate. 

8.5  Stabil  ity 

The  DCS  timing  subsystem  must  be  a stable  system  in  the  sense  of  a feedback 
control  system.  If  this  were  not  the  case,  the  nodal  clocks  would  be  subject  to  ever  increasing 
fluctuations  in  frequency  (to  the  limit  of  each  VCO's  control  range)  in  response  to  perturbations. 
The  result  would  be  buffer  overflows  and  continual  bit  slips.  The  stability  considerations  for 
each  of  the  four  timing  techniques  are  discussed  below. 

Independent  Clocks.  Since  every  nodal  clock  is  independent  of  every  other 
nodal  clock,  there  are  no  closed  timing  loops.  Thus,  this  technique  is  unconditionally  stable. 

Master- slave.  This  technique  is  unconditionally  stable  when  the  timing  reference 
to  each  nodal  clock  is  distributed  via  a fixed  master- slave  tree  network.  However,  there  is 
one  problem  that  can  occur  when  an  adaptive  timing  distribution  network  is  used.  The  config- 
uration of  the  timing  distribution  network  is  controlled  by  information  transmitted  between 
nodes.  Errors  in  this  data  could  cause  the  network  to  erroneously  configure  itself.  A stability 
problem  occurs  if  these  errors  cause  a portion  of  the  network  to  be  configured  as  a mutual  sync 
network.  For  example,  if  Node  A slaves  to  Node  B and  Node  B slaves  to  Node  A,  then  these 
two  nodes  form  a mutual  sync  network.  Since  the  loop  filters  in  the  nodal  synchronizer  have  not 
been  designed  for  mutual  sync  operation  (they  will  likely  be  type  two  loops)  the  synchronizers 
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at  nodes  A and  B and  will  become  unstable.  Fortunately,  there  is  a rather  simple  but  important 
remedy  for  this  situation,  i.e.,  the  use  of  error  detection  or  error  correction  coding  on  this 
data  to  prevent  the  use  of  erroneous  data.  One  can  make  the  probability  of  accepting  erroneous 
data  as  low  as  desired  by  adding  the  appropriate  number  of  parity  bits  (by  using  error  detection 
one  can  make  the  probability  of  accepting  an  erroneous  message  equal  to  2 ^ at  low  signal-to- 
noise  ratios  by  adding  p parity  bits).  Assuming  the  use  of  coding,  then  even  on  the  rare 
occasions  when  erroneous  data  is  accepted,  the  error  will  be  detected  rather  quickly  when  the 
next  update  of  that  information  is  received.  Since  very  narrowband  loops  are  used,  the  VCO's 
will  not  be  pulled  very  far  during  this  period.  Thus,  there  is  no  chance  that  a serious  problem 
can  occur. 

Time  Reference  Distribution.  The  only  problem  that  can  occur  with  time  reference 
distribution  is  the  same  problem  that  can  occur  with  a master-slave  system,  i .e.,  the  adaptive 
timing  network  configuration  can  be  erroneously  configured  into  a mutual  sync  network.  Again, 
the  remedy  for  this  problem  is  the  use  of  sufficient  coding  to  prevent  the  acceptance  of  erro- 
neous data.  Thus,  there  is  no  chance  that  a serious  problem  can  occur. 

Mutual  Synchronization.  Constraints  on  system  parameters  to  produce  a stable 
mutual  synchronization  system  have  taken  two  forms.  First,  one  must  ensure  that  changes  in 
nodal  frequencies  at  some  nodes  must  not  proceed  too  far  before  they  affect  the  frequencies  at 
other  nodes.  This  results  in  the  bounds  on  gain/delay  products  (loop  gain  and  link  delay)  that 
have  been  derived  by  many  investigators  for  example  networks.  The  bound  will  depend  on  net- 
work configuration,  but  typically  the  upper  bound  on  gain/delay  product  is  on  the  order  of  1 . 
Since  very  narrowband  loops  will  be  used,  link  delays  of  many  seconds  would  be  required  before 
there  would  be  a stability  problem.  Thus,  in  a real  network,  one  does  not  have  to  be  concerned 
about  gain/delay  products.  The  other  constraint  to  ensure  stability  is  on  the  loop  filter.  The 
loop  filter  must  be  chosen  such  that  the  closed  loop  transfer  function  has  magnitude  less  than 
one  at  all  u (>  0.  When  this  criteria  is  satisfied,  all  jitter  frequencies  are  attenuated.  This 
is  rather  important  because  of  the  closed  loops  in  a mutual  sync  network.  If  any  jitter  frequency 
were  amplified,  then  it  could  cause  a perturbation  in  nodal  frequency  which  would  be  continually 
amplified  from  node  to  node  in  the  network  resulting  in  an  unstable  network.  With  a type  one 
loop  using  a simple  low  pass  filter,  the  stability  criterion  is  satisfied  if  one  ensures  that  the 
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damping  coefficient  £>j2/2.  Assume  that  for  safety  £ = ) is  chosen.  Then  the  only  way 
instability  could  result  is  if  the  loop  gains  at  several  nodes  increased,  thereby  making  £ < ~J  2/2. 
For  this  example,  the  loop  gain  would  have  to  double  for  this  to  happen.  A change  in  the  loop 
gain  by  a factor  of  two  is  certainly  within  the  realm  of  possibility,  either  through  human  error 
in  calibration  or  through  device  changes.  Before  implementing  such  a system,  one  should 
devise  a technique  for  detecting  when  the  gain  gets  out  of  the  tolerance  range  to  ensure  that 
instability  will  not  occur. 

In  summary,  we  find  that  the  independent  clock  technique  is  unconditionally  stable. 
In  addition,  the  master-slave  and  time  reference  distribution  techniques  can  have  stability 
problems  if  certain  errors  occur  in  control  data.  However,  this  problem  is  easily  fixed  by  the 
use  of  error  control  coding.  Thus,  we  would  rank  these  approaches  equally  with  regard  to 
stability.  By  contrast,  the  mutual  sync  technique  normally  has  closed  loops  in  the  network, 
and  to  achieve  stability  the  loop  gain  at  every  node  must  be  held  within  tolerance.  This  task 
may  not  be  easy  to  ensure,  given  the  possibilities  of  human  error  in  calibration  and  device 
changes.  Thus,  we  rank  mutual  synchronization  as  less  desirable  than  the  other  three  on  the 
basis  of  system  stability. 
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Moni  torabi  lity 


The  characteristic  of  monitorabi lity  is  the  level  to  which  the  timing  subsystem  can 
provide  data  used  in  assessing  its  own  performance.  It  is  apparent  that  moni torabi li ty  is  valuable 
at  least  to  the  extent  that  an  option  such  as  a redundant  piece  of  equipment,  an  alternate  link  or 
another  timing  source  is  available  and  can  be  utilized  as  the  result  of  decisions  made  on  the  basis 
of  monitored  information. 

Monitorabi  lity  is  a significant  characteristic  because  it  can  lead  to  greater  avail- 
ability through  providing  tolerance  to  failure.  If  potential  failures  can  be  identified  prior  to 
failures  actually  occurring,  these  situations  can  be  remedied  in  an  off-line  basis  or  more  rapidly 
by  knowing  the  nature  of  the  failure  before  it  occurs.  The  result  is,  therefore,  both  an  increase 
in  MTBF  and  a reduction  in  MTYR  and  therefore  a greater  availability.  The  parameter  that  must 
be  measured  is  the  difference  in  availability  due  to  monitoring. 

This  discussion  will  encompass  a comparison  of  the  four  timing  system  techniques,  a 
focus  on  the  time  reference  distribution  approach,  discussions  of  several  inferences  that  may  be 
derived,  a representative  fault  scenario,  the  relationship  of  moni  torabi  lity  to  availability  and 
some  side  benefits  derived  on  the  basis  of  monitorabi  lity . 

8.6.1  Comparison  of  Techniques 

The  four  techniques  of  independent  clocks,  master-slave,  mutual  synchronization 
and  time  reference  distribution  can  be  compared  with  regard  to  their  relative  moni  torabi  I i ty . It 
can  be  seen  that  all  four  techniques  have  a fundamental  identical  moni  torabi  lity  capability  in 
that  they  can  see  those  actions  that  take  place  within  a given  node  of  interest  and  at  the  periphery 
of  that  node.  Thus  things  as  the  characteristics  of  a node's  clock,  backup  clock,  frequency  dis- 
semination circuitry,  phase-locked  loop  operation,  modem  operation  and  receipt  of  signals  from 
adjacent  nodes  are  readily  available  through  any  of  the  four  timing  approaches.  A significant 
addition  exists  with  time  reference  distribution  (TRD). 

Generally,  with  the  exception  of  TRD  each  of  the  techniques  is  limited  to  data 
that  exists  within  a node  with  the  exception  of  some  hybrids  which  might  be  defined  wherein  data 
is  shared  between  nodes  of  the  system  as  it  is  in  TRD.  Therefore,  it  can  be  stated  that  all  other 
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techniques  fundamentally  have  the  same  degree  of  monitorabi lity  capability.  The  differences  in 
TRD  will  be  discussed  in  the  following  paragraph  which  sets  it  significantly  aside  and  above  the 
other  approaches  in  terms  of  monitorabi  lity  capability. 

8.6.2  Time  Reference  Distribution 

The  time  reference  distribution  technique  has  all  the  capability  of  the  other  tech- 
niques in  assessing  its  internal  nodal  operation.  In  addition,  it  has  the  ability  to  predict  impact- 
ing failures,  detection  of  external  stresses,  determination  of  out-of-specification  trends  and  the 
ability  to  isolate  faults. 

The  enhanced  monitorabi  lity  of  TRD  is  based  on  the  fact  that  data  is  shared  through- 
out the  network  between  all  of  the  nodes  in  the  TRD  case.  Adaptive  action  such  as  organization 
and  reorganization  provide  hints  of  other  failure  or  operational  activity  within  the  overall  network. 
Some  of  the  data  that  is  available  for  monitorabi  lity  purposes  includes  the  time  of  the  local  clock, 
time  differences  between  the  local  clock  and  adjacent  clocks,  measured  but  uncorrected  time 
differences  between  each  adjacent  node  and  its  ultimate  master,  measured  time  difference 
between  the  local  clock  and  its  ultimate  reference  via  each  adjacent  node,  propagation  time 
between  the  local  node  and  each  adjacent  node,  rank  of  all  nodes  and  demerits  of  all  paths  of 
the  network . 

There  are  a number  of  sources  wherein  the  above  data  is  derived  and  these  include 
buffer  status,  time  difference  between  transmit  and  receive  multiplexer  framing  epochs,  received 
frequency  values,  local  frequency  value,  local  time,  received  data  from  adjacent  nodes,  local 
node  rank,  connecting  path  demerits,  manual  inputs  possibly  inserted  to  diagnose  the  system, 
update  counter  contents,  the  rank  and  identification  of  each  ultimate  reference  associated  with 
either  the  node  in  question  or  its  adjacent  neighbors,  accumulated  path  demerit,  failed  nodes  and 
links  and  overall  network  status.  These  sources  provide  a picture  of  the  overall  network  operation 
especially  from  a timing  standpoint  at  any  given  node  in  a TRD  scheme.  The  analysis  job  is  to 
look  for  unexpected  changes  from  time  frame  to  time  frame  and  assess  their  importance.  In 
addition  to  the  discrete  values  of  above  sources  of  data,  sets  of  information  are  available  by 
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gathering  groups  of  information  such  as  link  time  delays  to  all  adjacent  nodes  or  to  all  adjacent 
nodes  associated  with  each  nearest  adjacent  node. 

Some  of  the  information  that  can  be  important  in  assessing  TRD  operation  and 
predicting  network  malfunctions  which  can  lead  to  timing  system  malfunctions  include  the 
information  in  Table  8.6.2. 

Another  factor  in  addition  to  the  above  is  that  of  the  occurrence  and  magnitude  of 
the  clock  corrections.  The  subscripts  a and  b are  used  to  denote  pairs  wherein  adjacent  nodes 
may  number  from  anything  up  to  roughly  10  nodes,  each  having  a set  of  values  corresponding  to 
items  listed  in  Table  8.6.2.  Examples  of  sets  would  be  all  of  the  buffer  contents  at  the 
adjacent  nodes  referred  to  the  given  node  in  question  or  all  of  the  buffer  contents  at  a given  node 
for  the  links  connected  to  the  adjacent  nodes. 

Therefore,  in  summary,  it  has  been  noted  that  such  things  as  the  status  of  all 
connecting  buffer,  links,  and  update  counters  can  be  made  available  at  any  given  node  in  the 
TRD  system.  From  this  data  numerous  inferences  can  be  made  as  to  just  what  is  happening 
within  the  network. 

8.6.3  Inferences 

A few  representative  inferences  based  on  data  that  can  be  accumulated  by  a TRD 
node  will  be  discussed  in  this  paragraph. 

In  the  event  of  a change  of  the  ultimate  reference,  an  inference  could  be  made 
that  1)  the  original  ultimate  reference  failed,  2)  failure  or  change  of  a path  or  node  connecting 
the  original  ultimate  reference  has  occured,  3)  activation  of  a better  path  to  a new  ultimate 
reference  has  been  determined,  or  4)  activation  of  a better  ultimate  reference  has  occurred. 

Another  inference  would  occur  when  a change  of  adjacent  node  from  which  timing 
is  derived  occurs  and  this  might  denote  l)all  possibilities  in  the  previous  example,  2)  failure  or 
change  in  the  path  to  the  adjacent  node  originally  used,  3)  activation  of  a new  best  adjacent 
reference  or  restoral  of  a prior  failed  condition,  or  4)  periodic  maintenance  outage  is  occurring. 

A third  inference  can  be  derived  if  a change  in  path  delay  or  measured  time  difference  over  any 
one  or  more  connecting  links  should  occur,  then  it  might  denote  that  1)  frequency  or  phase  change 


8-28 


. 

* 


*» 

* 


ab 


ab 


ab 


ba 


'ba 


ub 


u u 
u-b 

N, 


ba 


ba 


Table  8.6.2.  Available  Data 
buffer  contents  - connected  to  b 
buffer  fill/empty  rate  at  a connected  to  b 
multiplexer  frame  time  difference  measured  at  a re  b 
frequency  received  from  n 
local  clock  frequency 
local  time 
local  node  rank 

demerit  of  path  between  a and  b 
update  counter  contents  at  a 

multiplexer  frame  time  difference  measured  at  b re  a 
rank  of  node  b 

ultimate  reference  node  for  node  b 

rank  of  ultimate  reference  node  for  node  b 

path  demerit  - node  b to 

identity  of  failed  node 

identity  of  failed  link 

update  counter  contents  at  b 

buffer  contents  at  b 

time  at  b 

buffer  contents  for  the  link  at  node  b connected  to  a 
rate  of  change  of 


i 
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of  a clock  or  clocks  along  the  path  to  the  ultimate  reference  has  occurred;  2)  alternate  routing 
has  occurred  due  to  a network  switching  assignment;  3)  link  equipment  has  failed,  or  changed,  or 
4)  spoofing  is  being  encountered. 

Several  other  areas  from  which  one  might  derive  inferences  are  assessments  of  demerit 
values  and  changes  thereto,  update  counter  activity  and  contents,  buffer  activity  and  contents, 
a change  occurring  in  "local  time"  value,  rank  changes  and  variations  in  frequency  values.  A 
synergism  of  available  data  could  be  used  through  applying  techniques  of  predictive  analysis  to 
groups  of  data  available. 


8.6.4 


Example  Fault  Scenario 


A somewhat  typical  fault  scenario  will  be  briefly  discussed  to  show  simply  how 
network  failures  can  be  detected  at  a given  node  if  TRD  is  utilized.  Reference  is  called  to 
Figure  8.6.4  which  shows  two  network  configurations  and  timing  source  paths  both  before  and 
after  failures  have  occurred.  The  failures  are  denoted  by  the  x's  on  the  links  between  Nodes  9 
and  13  and  12  and  14.  When  the  failures  are  introduced  in  a TRD  based  timing  system  the  network 
must  reorganize  itself  per  the  reorganization  rules  to  a configuration  possibly  like  that  shown  in 
the  second  part  of  the  figure  which  denotes  the  case  after  reorganization.  At  Node  i data  is 
available  or  can  be  made  available  as  to  all  of  the  activity  in  the  network  through  its  adjacent 
Nodes  1,2,  3 and  4 gathering  data  as  discussed  in  earlier  paragraphs  on  monitorability.  It  is 
possible  to  deduce  just  what  has  occurred  and  be  prepared  should  a trend  develop  in  failures. 


8.6.5 


Avai  lability  Impact 


The  characteristic  of  monitorability  is  only  important  when  it  can  be  used  to  enhance 
the  availability  of  the  system.  This  comment  applies  to  the  timing  system  as  such  and  not  to  other 
benefits  of  having  a very  monitorable  approach.  It  is  possible  to  enhance  availability  through 
increasing  the  time  between  failures  and  reducing  the  time  it  takes  to  restore  operation  after  a 
failure  has  occurred.  It  is  possible  to  determine  whether  various  frequencies  in  the  network  are 
drifting  out  of  specification  frequency-wise  and  switch  to  backup  thus  avoiding  a failure  where 
one  would  have  occurred  should  such  action  not  have  been  taken.  If  a failing  time  source  is 
being  used  it  is  possible  to  switch  to  an  alternate  source  ahead  of  time  avoiding  a failure. 


I 


Also,  switching  in  any  available  redundancy  to  heal  a sensed  failure  or  potential  failure  can 
enhance  the  overall  availability.  The  detection  of  spoofing  and  developing  strategies  to  avoid 
it  can  be  beneficial  from  an  availability  standpoint.  The  restoral  time  required  to  reactivate  a 
network  that  would  have  failed  can  be  minimized  through  alerting  maintenance  personnel  ahead 
of  time  to  the  nature  of  a potential  failure  and  thus  minimizing  MTTR.  Selective  redundancy 
switching  can  also  enhance  availability. 

The  data  discussed  in  reference  to  TRD  is  available  on  essentially  a free  basis  for 
monitorabi lity  purposes.  Much  of  the  data  mentioned  is  required  just  to  implement  TRD  as  a 
timing  scheme . 

8.6.6  Benefits 

In  addition  to  enhancing  the  availability  of  the  timing  subsystem,  side  benefits 
will  evolve  that  can  apply  to  the  overall  DCS  network  SYSCON.  The  synergism  of  the  many 
data  available  can  be  used  to  predict  other  system  failures  and  restoral  thereof  not  directly 
associated  with  timing  subsystem.  Although  this  is  not  fundamentally  important  to  the  decision 
of  a timing  approach,  it  comes  along  pretty  well  free  of  charge  and  can  be  made  use  of  by  the 
DCS  thus  enhancing  the  desirability  of  the  time  reference  distribution  approach. 

8.6.7  Conclusions 

It  is  evident  that  a time  reference  distribution  technique  is  far  superior  to  the  other 
approaches  when  the  characteristic  of  monitorabi  lity  is  considered.  Also,  it  is  projected  that 
the  cost  of  obtaining  data  for  monitorabi li ty  is  essentially  negligible  since  the  information  already 
resides  in  the  system  when  TRD  is  implemented.  The  cost  that  would  ensue  would  be  the  storage 
over  a period  of  time  of  historical  quantities  of  the  information  such  that  inferences  may  be  made 
from  the  data.  Obviously  there  would  be  some  slight  cost  in  addition  to  basic  TRD  operation 
which  in  itself  only  requires  a limited  portion  of  the  data  available. 
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8.7  Susceptibility 

Jammer  Strategy 

A major  attribute  of  a communications  network  is  its  connectivity.  Thus,  an 
antagonist  would  like  to  disconnect  the  network  into  some  number  of  partitions.  He  may  attempt 
to  do  this  through  physical  destruction  of  parts  of  the  network  or  by  producing  interfering  signals 
(jamming).  In  case  the  network  facilities  are  heavily  protected,  in  a physical  sense,  the  antag- 
onist may  concentrate  much  of  his  effort  toward  jamming  selected  portions  of  the  network.  He 
will  select  those  portions  which  he  believes  will  produce  the  most  disturbance.  Hopefully,  from 
the  standpoint  of  the  network,  the  jammer  will  not  be  able  to  actually  disconnect  the  network 
due  to  inability  to  locate  his  jamming  emitters  in  the  most  strategic  geographical  positions.  Thus, 
he  may  concentrate  on  jamming  selected  portions  of  the  network  in  such  a manner  as  to  disrupt 
some  vital  network  subsystem  such  as  the  timing  subsystem  of  the  TDM  network.  The  different 
timing  techniques  may  be  affected  by  different  amounts  depending  on  the  strategy  used  by  the 
jammer.  The  independent  clocks  technique  is  less  susceptible  to  disruption  by  a jammer  than  any 
of  the  other  timing  techniques.  In  fact,  under  extreme  jamming  conditions,  the  network  utilizing 
other  techniques  will  appear  to  tend  to  revert  to  an  independent  clocks  configuration  when  viewed 
externally. 

Types  of  Information 

Each  node  of  the  timing  subsystem  obtains  essentially  two  types  of  information  from 
other  nodes  of  the  network.  These  include  timing  information  and  control  information. 

Timing  Information  - For  master-slave  and  mutual  sync  timing  information  is  con- 
tained in  the  bit  transitions  of  each  data  stream.  As  the  jamming  intensity  increases  from  zero 
to  the  point  at  which  the  link  is  lost,  the  timing  jitter  increases  until  no  useful  timing  information 
can  be  extracted  with  available  equipment  from  the  contaminated  data  stream.  It  is  not  antici- 
pated that  techniques  can  be  developed  to  significantly  reduce  the  effects  of  jamming  on  timing 
information  for  all  levels  of  jamming.  However,  monitoring  of  channel  signal-to-noise  ratio  can 
give  some  indication  of  the  timing  information  quality  obtainable  at  a given  period  from  each 
incoming  link.  This  can  aid  in  selecting  the  best  timing  reference  at  each  node. 
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Control  Information  - Adaptive  reorganization  schemes  as  well  as  fault  detection 
and  isolation  apparatus  at  each  node  may  use  control  information  obtained  from  remote  nodes. 
Control  information  will  most  probably  be  transmitted  between  nodes  via  overhead  channels 
(orderwire).  These  channels  may  be  protected  to  some  extent  by  error  detection  and/or  forward 
error  correction  encoding.  Thus,  the  problem  of  incorrect  control  information  can  almost  be 
avoided  at  the  expense  of  possibly  no  new  updated  control  information  during  those  periods  of 
most  intense  active  jamming.  Aside  from  error  detection  and  correction  encoding  of  control 
information  messages,  the  overhead  channels  are  susceptible  to  the  same  types  of  jamming  as  the 
data  channels. 

Adaptive  Reorganization 

An  intermittent  jammer  on  selective  links  may  cause  the  reorganization  scheme  to 
remain  in  a constant  state  of  reorganization  because  there  is  a time  lapse  between  a network  fault 
and  the  actual  recognition  at  the  affected  node  of  the  faulty  condition.  There  is  then  a further 
time  delay  in  selecting  a new  reference  and  becoming  frequency  and  phase  locked  to  the  new 
reference.  This  causes  a transient  in  that  portion  of  the  timing  tree  below  the  first  affected  node. 
Concomitantly,  nodes  in  other  timing  chains  that  are  communicating  with  nodes  in  the  disturbed 
chain  will  experience  effects  of  the  disturbance.  Upon  removal  of  the  jamming  from  a selected 
link,  the  adaptive  reorganization  scheme  may  decide  to  again  select  this  link  for  obtaining  its 
reference  and  thereby  cause  another  transient  in  part  of  the  network.  Thus,  it  is  easily  envisioned 
that  a concerted  attack  by  a small  set  of  intermittent  jammers  might  be  able  to  induce  the  network 
to  go  into  a perpetual  state  of  reorganization.  In  order  to  reorganize  properly,  each  node  must 
have  an  adequate  amount  of  correct  control  information.  If  this  is  not  the  case,  then  one  of  the 
following  may  happen: 

1 . The  network  timing  subsystem  may  become  partitioned 

2.  Portions  of  the  network  may  become  configured  as  mutual  sync 

3.  Portions  of  the  network  may  not  sync  to  its  best  available  reference 

If  none  of  the  above  happens,  the  subsystem  will  still  tend  to  revert  to  an  independent  clocks 
system.  This  is  true  because  during  the  reorganization  process,  the  individual  nodes  must  revert 
to  self-reference. 
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Aside  from  e ncoding  of  the  control  information  one  possible  counteraction  against 
the  intermittent  jammer  is  for  each  node  to  maintain  a running  monitor  of  the  links  in  question 
and  refrain  from  referencing  any  of  those  whose  recent  history  of  outages  conform  to  the  inter- 
mittently jammed  conditions  described  above.  Both  master-slave  and  TRD  may  be  configured  to 
utilize  an  adaptive  reconfiguration  scheme. 

It  is  believed  that  coding  of  control  information  and  careful  monitoring  of  links  at 
each  node  to  detect  the  intermittent  jamming  strategy  can  be  used  to  virtually  eliminate  the 
dangers  to  the  timing  subsystem  via  the  adaptive  reorganization  scheme  discussed  above. 

Master-Slave  and  TRD 

The  master-slave  and  TRD  techniques  are  similar  in  that  timing  information  is 
dissiminated  from  a master  or  ultimate  reference  node  to  all  other  nodes  in  the  network  via  a tree 
structure  of  links  and  nodes.  The  major  conceptual  difference  is  that  the  TRD  technique  attempts 
to  remove  the  uncertainties  associated  with  link  delay  variations  by  taking  advantage  of  two-way 
timing  information  whereas  the  master-slave  merely  averages  the  one-way  timing  information. 
Thus,  loss  of  links  due  to  jamming  affects  both  techniques  in  the  same  manner  but  the  total  effect 
can  be  different  depending  on  how  closely  each  can  maintain  the  individual  nodes  instantaneous 
frequencies  to  each  other.  In  either  case,  changing  references  following  degradation  or  loss  of 
a link  due  to  jamming  causes  a phase  and  frequency  transient  in  the  affected  nodal  clock.  This 
transient  is  coupled  to  all  nodes  in  the  chain  of  the  tree  below  the  affected  node. 

Mutual  Sync 

The  mutual  sync  configuration  will  experience  a transient  when  a link's  contribution 
to  the  total  phase  error  is  removed  if  that  link  is  being  jammed  and  again  if  the  link  is  reinserted. 
These  disturbances  will  cause  a change  in  network  frequency.  Thus,  a concerted  effort  by  inter- 
mittent jammers  may  set  up  a seesaw  movement  of  system  frequency.  Such  disturbances  may  be 
accentuated  by  the  mutual  sync  network  because  of  the  longer  time  constants  associated  with  the 
oveiall  network  when  compared  with  that  of  individual  nodes  and  the  larger  swings  on  buffer  con- 
tents at  individual  nodes  when  compared  with  master-slave.  The  network's  response  to  each  dis- 
turbance (loss  or  reinsertion  of  links  following  jamming  activation/deactivation)  may  vary  con- 
siderably depending  on  where  It  occurs  in  the  network.  . In  general,  the  disturbance  should  be 


more  pronounced  if  it  occurs  at  nodes  that  are  highly  connected  to  the  remainder  of  the  network, 

i.e.,  nodes  near  the  connectivity  center  of  the  network.  Disturbances  occurring  at  fringe  nodes 
should  have  a lesser  affect  on  the  overall  network  than  those  occurring  near  the  center  of  the 
network.  Nevertheless,  it  appears  that  such  disturbances  in  the  mutual  sync  technique  propagate 
further  and  require  a longer  period  to  settle  out  than  for  other  techniques. 

TRD,  master-slave  and  Mutual  Sync  can  monitor  the  phase  error  between  the  local 
clock  and  each  of  its  actual  or  potential  reference  sources.  Comparisons  can  be  made  between 
these  errors  and  abnormal  conditions  resulting  from  jammer  strategy  may  be  detectable  from  these 
comparisons.  The  cost  of  implementation  of  such  a scheme  is  not  believed  to  differ  significantly 
among  the  three  disciplined  techniques  since  each  technique  must  provide  buffers  between  the 
local  node  and  each  communicating  link.  However,  it  is  believed  that  this  scheme  would  be 
most  effective  with  the  TRD  technique  and  least  effective  with  the  mutual  sync  scheme  with 
master-slave  somewhere  in  between.  The  reason  for  this  is  due  to  the  amount  of  deviation  in 
nodal  clock  error  phases  that  is  expected  to  occur  throughout  the  network.  It  is  expected  that 
the  normal  amount  of  deviation  for  TRD  will  be  considerably  less  than  for  master-slave  since  TRD 
attempts  to  remove  the  affect  of  link  disturbances  as  well  as  intermediate  node  perturbations. 
Furthermore,  the  amount  of  deviation  in  master-slave  phase  errors  is  less  than  that  for  mutual  sync. 
The  simulation  results  of  Section  7.0  support  these  conjectures. 

In  conclusion,  the  timing  techniques  may  be  ranked  from  least  susceptible  to  most 
susceptible  to  jamming  (with  or  without  strategy)  as  follows: 

1.  Independent  C locks 

2.  TRD 

3.  Master-Slave 

4.  Mutual  Sync 

8.8  Flexibility 

Flexibility  has  been  defined  as  the  impact  on  orderly  implementation  growth  and 
extension  of  the  future  synchronous  switched  digital  DCS.  It  would  appear  that  many  other 
factors  have  a substantially  greater  effect  on  the  flexibility  of  the  DCS  than  does  the  choice  of 


timing  subsystem.  Furthermore,  there  is  nothing  inherently  problematic  from  a flexibility  point 
of  view  in  any  of  the  timing  techniques.  Each  technique  can  be  implemented  on  a piece  meal 
basis.  Each  technique,  once  implemented,  can  be  extended  to  provide  timing  to  additional  nodes 
in  different  geographical  areas.  Obviously  a detailed  analysis  would  be  required  once  system 
specifics  are  better  defined.  However,  from  a broad  overview  based  on  the  fundamental  differences 
in  the  timing  techniques,  there  is  no  reason  from  a flexibility  point  of  view  to  favor  any  one  tech- 
nique over  any  other. 

8.9  Interoperability 

The  characteristic  to  be  evaluated  is  whether  the  application  of  any  of  the  candi- 
date timing  techniques  to  the  DCS  will  substantially  facilitate  the  interface  between  the  DCS  and 
other  communications  systems.  The  specific  problem  of  concern  exists  at  the  DCS  node  that  trans- 
mits and  receives  data  streams  from  both  a node  external  to  the  DCS  and  other  DCS  nodes.  It 
seems  apparent  that  the  clock  at  such  a node  should  in  no  way  be  influenced  by  the  received  bit 
rate  on  the  interface  link.  Thus,  the  amount  of  buffering  required  on  such  a link,  which  is  a 
measure  of  interoperability  from  a timing  point  of  view,  is  directly  related  to  the  difference  in 
clock  frequencies  at  the  two  nodes  on  either  side  of  the  interface  link.  The  timing  technique 
that  holds  promise  for  minimizing  this  frequency  difference  would  be  preferable.  This  question  is 
not  unlike  one  of  the  questions  addressed  in  the  compatibility  of  the  CDS  timing  subsystem  with 
other  global  timing  systems.  Again,  one  is  concerned  with  the  possibility  of  common  timing  for 
separate  systems;  and  again,  the  answer  is  the  same.  Independent  clocks  by  definition  preclude 
any  advantage  that  would  accrue  as  a result  of  common  disciplining.  Mutual  synchronization 
offers  no  relief  because  of  limited  connectivity  between  systems.  Master-slave  and  time  reference 
distribution  have  the  potential  of  supporting  timing  commonality  between  systems  as  a result  of  their 
hierarchial  structures.  However,  the  practicality  and  probability  of  a common  reference  is  much 
greater  in  a time  reference  distribution  system  where  the  ultimate  reference  would  be  UTC. 


Naturally,  greater  interoperability  would  not  result  if  systems  interfacing  with  the  DCS  chose 
not  to  reference  their  timing  to  UTC;  however,  based  on  the  potential  that  exists,  time  reference 
distribution  is  determined  to  be  the  best  technique  from  an  interoperability  point  of  view.  There 
is  little  to  separate  the  other  techniques. 
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Availabi  lity 


Availability  is  the  quotient  of  mean-time-between-fai lures  (MTBF)  and  the  sum  of 
MTBF  and  the  mean  time  to  repair  or  restore  (MTTR).  Availability  is  a significant  characteristic 
for  all  major  systems  and  its  importance  is  apparent  for  the  timing  subsystem  by  nature  of  the 
misssion  of  the  DCS. 

In  order  to  provide  an  availability  number  for  each  timing  subsystem  alternative, 
MTBF  and  MTTR  would  have  to  be  calculated  for  each.  This  calculation  would  follow  a paper 
design  of  the  alternatives  and  therefore  the  accuracy  of  the  availability  estimates  would  be  a 
function  of  the  level  of  detail  undertaken  in  the  paper  design.  However,  it  is  significant  to 
note  that  only  the  relative  availability  of  the  approaches  is  of  significance  in  the  elevation  of 
timing  approches  rather  than  the  cbsolute  availabilities.  The  evaluation  of  relative  availability 
appears  to  be  achievable  with  less  detailed  designs  than  the  detail  required  to  derive  absolute 
availability.  Additional  availability  can  be  built  into  any  of  the  timing  subsystem  alternatives. 
Relative  inherent  availability  with  minimum  redundancy  for  the  alternatives  is  most  desirable 
from  a cost-effectiveness  standpoint. 

In  this  section  the  common  and  unique  aspects  of  each  of  the  four  timing  approaches 
will  be  investigated  and  discussed  with  respect  to  their  impact  on  relative  availability. 

8.10.1  Comparison  of  Techniques 

Time  reference  distribution,  mutual  synchronization,  master-slave  and  independent 
clock  timing  approaches  are  discussed  in  the  following  paragraphs. 

8.10.1.1  Common  Factors 

In  assessing  the  various  timing  approaches,  several  constituent  elements  of  each 
timing  approach  at  a given  node  are  considered  common  to  all.  These  include  buffers,  clocks, 
frequency  synthesizers  and  phase- locked  loops.  The  buffers  in  some  cases  may  have  more  or  less 
bits  in  buffer  capabi  lity,  but  general  ly  their  avai lability  will  be  roughly  equal . These  several 
techniques  call  for  different  degrees  of  complexity  in  their  clocks,  and  this  could  be  considered 
a factor  in  their  relative  availability,  but  it  is  a rather  tenuous  one  to  quantify.  A brief  discuss- 
ion will  be  included  later  in  this  section. 
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8.10.1.2  Time  Reference  Distribution 


The  time  reference  distribution  approach  has  several  unique  elements  that  apply 
to  its  relative  availability  compared  with  the  other  approaches.  Also,  some  devices  that  are 
inherent  in  all  timing  approaches  impact  the  TRD  node  uniquely  and  will  be  discussed  here.  First 
the  link  selection  processor  is  that  element  which  conducts  the  evaluation  of  the  organization  and 
reorganization  process  for  a given  TRD  node.  The  time  interval  measurement  unit  is  unique  to 
TRD  in  that  it  is  the  means  whereby  received  time  epoch  is  referred  to  the  station  time.  The  clock 
error  processor  is  also  unique  to  TRD  in  that  it  provides  a calculated  value  for  clock  compensation 
which  is  the  result  of  shared  information  between  the  TRD  node  and  one  or  more  of  its  adjacent 
neighboring  nodes. 

The  highest  level  multiplexer  and  demultiplexer  uniquely  impact  TRD  in  that  they 
are  utilized  to  initiate  the  identification  of  the  station  epoch  to  the  neighboring  node  and  to 
interpret  the  received  epoch  for  the  purpose  of  calculating  time  difference  between  adjacent  nodes 
and  the  TRD  node.  The  modems  on  the  receive  side  of  each  interconnecting  link  are  important 
both  for  their  reception  of  the  carrier  which  provides  the  determination  of  frequency  and  the 
receipt  of  data  which  is  utilized  in  the  process  of  sharing  information  between  the  various  nodes 
of  a TRD  oriented  timing  network. 

8.10.1.3  Mutual  Synchronization 

In  the  case  of  a mutual  sync  node,  the  weighting  and  sum  circuitry,  control,  and 
multiple  phase  detectors  represent  unique  elements  applicable  to  the  unique  or  relative  availabil- 
ity of  this  timing  approach  compared  with  the  others.  Also,  the  receive  modems  are  important 
in  that  they  must  be  capable  of  providing  carrier  output  for  the  purpose  of  ascertaining  the  fre- 
quency characteristics  of  the  received  links.  Differing  from  the  TRD  modem  impact  is  that  data 
is  not  really  important  in  a pure  mutual  sync  approach. 

8.10.1.4  Master-Slave 

The  unique  aspects  of  the  master- slave  timing  node  are  its  control  and  the  single 
phase  detector  utilized  in  tracking  the  received  timing  signal.  Also,  the  carrier  detection  of 
the  modems  represents  an  availability  oriented  need. 
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8.10.1.5  Independent  Clocks 


Based  on  the  statement  that  the  clocks  have  a unique  and  very  difficult  to  deter- 
mine impact  on  relative  availability,  the  independent  clock  nodes  would  have  no  other  impact- 
ing unique  relative  availability  subsystems. 

8.10.2  Subjective  Evaluation 

In  order  to  ascertain  qualitatively  the  relative  availability  of  the  various  timing 
approaches,  a scale  of  one  to  10  is  posed.  This  scale  is  defined  as  least  to  most  complex  wherein 
the  complexity  should  impact  both  MTBF  and  MTTR  and,  thus,  the  inherent  availability  on  a 
relative  basis  for  each  of  the  approaches.  The  complexity  factors  for  the  various  approaches  have 
been  assigned  as  follows.  Independent  clocks  are  assigned  a complexity  factor  of  unity,  master- 
slave  a factor  of  five,  mutual  synchronization  a factor  of  seven,  and  TRD  a factor  of  10.  Thus 
it  is  seen  that  TRD  being  more  complex  of  the  selected  approaches  for  evaluation  most  likely 
represents  the  lowest  relative  availability  of  the  bunch.  The  spread  of  relative  availabilities  in 
the  actual  terms  probably  isn't  extremely  significant  among  the  master-slave,  mutual  synchroni- 
zation, and  TRD  approaches.  However,  the  independent  clock  approach  which  is  basically 
autonomous  should  allow  the  best  relative  availability.  The  type  of  clock  used  certainly  must 
have  some  impact  on  the  relative  availability.  Although  the  cesium  beam  standards  have 
reported  MTBF's  of  25,000  hours,  the  other  clocks,  namely  rubidium  and  crystal,  represent  quite 
reliable  mechanisms.  Therefore,  the  spread  should  not  be  extremely  broad  in  the  availabilities  of 
each  of  these  clocks.  If  anything,  the  various  clocks  impacts  can  be  related  on  a qualitative  basis 
with  a scale  of  one  to  10  reflecting  least  reliability  to  most  reliable.  An  evaluation  would  show 
this  to  be  unity  for  independent  clocks,  10  for  master-slave,  five  for  mutual  synchronization,  and 
10  for  time  reference  distribution.  The  assumption  above  is  that  independent  clocks  will  require 
the  most  stringent  and  complex  clock,  whereas  master-slave  and  TRD  require  a fairly  loose  clock, 
and  mutual  sync  in  between.  It  is  possible  to  use  the  best  clocks  in  all  systems,  thus  equalizing 
the  relative  impact  of  clocks,  but  unfortunately,  impacting  the  cost-effectiveness  of  the  system 
implementation  in  each  case. 


8-40 


V ****<• 


t 


9.0  RANKING  OF  THE  DESIRABLE  CHARACTERISTICS 

Early  in  the  study,  two  efforts  resulted  in  1)  the  definition,  discussion,  and 
adoption  of  the  various  timing  schemes  to  be  applied  in  combination  or  individually  to  provide 
the  Network  Timing  and  Synchronization  function,  and  2)  the  key  system  characteristics  that 
would  be  used  for  the  relative  evaluation  of  the  performance  of  the  timing  schemes  in  the  context 
of  certain  scenarios  and  nodal  configurations. 

In  this  section,  the  relative  importance  of  each  of  these  aforementioned  charac- 
teristics is  addressed.  These  results,  when  combined  with  the  results  of  Section  8.0  will  then 
enable  one  to  arrive  at  a figure  of  relative  worth  or  utility  of  the  timing  subsystem  being  evaluated. 

The  Desirable  Characteristics  Are  Reviewed  Below 

Survivability 

The  degree  to  which  the  timing  subsystem  continues  to  perform  its  function  (to 
provide  network  timing)  during  periods  of  stress.  The  measure  of  survivability  is  slip  rate  which 
is  directly  proportional  to  the  frequency  offset  between  nodal  clocks.  If  offset  increases  in  a 
stressed  environment,  bit  slips  increase.  Slip  rate  may  be  defined  as  the  number  of  times  per 
unit  that  buffers  must  be  reset.  This  is  a factor  which  is  directly  proportional  to  the  rate  of  phase 
error  between  the  nodal  clocks.  To  evaluate  this  parameter  we  must  measure  the  rate  of  accumu- 
lation of  phase  error  between  nodal  clocks  under  a variety  of  possible  scenarios. 

Error  Propagation 

To  prevent  disturbances  (such  as  stresses  or  path  delov  variations)  in  one  part  of 
the  network  from  propagating  and  influencing  the  phase  of  nodal  clocks  in  other  parts  of  the 
network.  The  measure  of  error  propagation  is  the  offset  in  phase  of  each  nodal  clock  from  its 
steady-state  value  as  a result  of  network  stresses. 

Compatibility  With  Other  Global  Timing  Systems 

The  degree  with  which  the  DCS  network  frequency  and/or  precise  time  agrees  with 
that  provided  by  other  global  timing  systems  such  as  Loran  C,  Omega  and  GPS.  Compatibility 
will  be  evaluated  by  measuring  the  additional  capability  provided;  the  greater  availability  and 
survivability  of  the  timing  subsystem  and  the  greater  availability  of  precise  time. 
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Precise  Time  Availability 

Precise  time  can  be  defined  as  the  correct  absolute  time  as  kept  by  a standards 
laboratory  such  as  the  USNO.  The  characteristic  to  measure  is  the  availability  of  and  accuracy 
of  precise  time. 

Stability 

Stability  is  defined  in  terms  of  feedback  control.  Either  perturbations  die  out  with 
time  and/or  distance,  in  which  case  the  transient  interval  is  of  importance,  or  the  system  degrades 
with  increasing  time,  thus  constituting  a failure. 

Monitorability 

The  level  to  which  the  timing  subsystem  can  provide  data  used  in  accessing  its 
own  performance.  The  parameter  that  must  be  measured  is  the  difference  (increase)  in  system 
availability  due  to  self-monitoring. 

Susceptibility 

Susceptibility  is  the  degree  to  which  the  timing  subsystem  may  be  spoofed,  jammed 
or  disrupted  by  an  electrical  means  as  well  as  disrupted  due  to  natural  phenomena.  Slip  rate  will 
again  be  the  measurable  quantity. 

Flexibility 

Flexibility  is  the  degree  to  which  the  timing  subsystem  is  compatible  with  the 
orderly  implementation,  growth  and  extension  of  the  future  synchronous  switched  digital  DCS. 

Interoperability 

Interoperability  is  the  degree  to  which  the  timing  subsystem  influences  the  traffic 
interface  between  DCS  and  other  military  and  commercial  communication  systems.  Interopera- 
bility will  be  quantified  in  terms  of  slip  rate  at  the  interfaces.  It  is  directly  proportional  to  the 
rate  of  accumulation  of  phase  between  the  timing  subsystems  of  the  interfacing  systems. 
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Availability  is  a defined  quantity  and  equals  MTBF  divided  by  the  sum  of  MTBF 


and  MTTR. 

This  is  no  unique  approach  to  use  as  a guide  in  a characteristics  sensitivity  study, 
i.e.  (the  sensitivity  of  system  utility  to  the  absence  or  presence  of  each  of  the  characteristics) 
and  the  approach  can  range  from  a rigorous  mathematical  treatment  to  purely  qualitative  judge- 
ment. As  will  be  pointed  out,  the  approach  planned  for  this  phase  of  the  effort  lies  between 
these  two  extremes  hopefully  taking  advantage  of  mathematical  rigor  where  rigor  is  possible  and 
enlightened  qualitative  judgement  where  mathematical  rigor  is  not  possible  or  advisable. 

Such  a study  would  be  relatively  amenable  to  mathematical  rigor  if  certain 
assumptions  could  be  justified.  Among  these  are: 

] . The  parameters  employed  in  the  sensitivity  study  must  be  measurable  without 
ambiguity.  Many  of  the  parameters  above  can  be  measured  as  defined  and 
evaluated  on  a quantitative  basis;  however,  others  require  qualitative  judge- 
ment for  evaluation.  The  requirement  for  qualitative  judgement  does  not 
invalidate  the  parameter  or  reduce  its  importance  to  the  study.  It  merely 
points  out  the  difficulty  of  measurement  of  some  important  parameters. 


2.  The  parameters  employed  in  the  sensitivity  study  must  be  independent  or 
correlatable  in  a rigorous  manner.  Monitorability  can  be  considered  an 
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independent  parameter;  however,  the  measurement  of  monitorability  is  a 
change  in  availability  resulting  from  the  benefits  of  monitorability.  Con- 
siderable judgemen*  will  be  required  to  evaluate  how  monitorability  extrap- 
olates into  availability.  Again,  monitorability  and  flexibility  are  two  very 
important  characteristics  of  the  system.  They  do  not  lend  themselves  to  a 
mathematically  rigorous  approach  in  a characteristics  sensitivity  study  however. 
Survivability  is  also  a very  important  criterion  for  the  system,  yet  survivability 
results  from  a very  complex  combination  of  other  characteristics.  Thus,  it 
cannot  be  claimed  that  the  characteristics  are  - or  should  be  in  this  particu- 
lar case  - independent.  As  for  "correlatable"  in  a mathematical  sense,  con- 
siderable subjective  judgement  would  be  involved  in  this  approach  as  well. 
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3.  The  parameters  employed  in  the  sensitivity  study  must  constitute  a complete 
set,  a fact  which  is  contrary  to  ease  of  mathematical  assessment. 

4.  Success  or  failure  of  the  system  must  be  evaluated  in  terms  of  the  mission  of 
the  system.  The  mission  of  the  NTSS  is  intimately  related  to  the  mission  of 
the  DCS  and  will  remain  basically  the  same  in  all  scenarios.  All  parameters 
chosen  for  evaluation  of  the  various  NTSS  schemes  have  an  effect  on  the 
accomplishment  of  the  mission;  however,  the  relative  weight  to  apply  to 
subsets  of  the  overall  misssion,  and  thereby  to  specific  characteristics  of  the 
NTSS,  is  a complex  function  of  employment  - peacetime/wartime,  CONUS/ 
OCONUS  - and  a host  of  other  variables.  The  evaluation  is  further  compli- 
cated by  the  fact  that  the  mission  statement  for  the  DCS  will  not  be  static 
but  will  change  and  evolve  with  time. 

From  the  above,  we  conclude  that  a rigorous  mathematical  trade-off  analysis  of 
the  various  NTSS  schemes,  based  upon  a rigorous  mathematical  characteristics  sensitivity  study 
is  not  an  advisable  route  to  pursue.  Numerical  results  could  be  obtained  which,  on  the  surface, 
would  imply  mathematical  rigor  but  which  would  in  fact  be  based  upon  subjective  inputs  of 
questionable  validity.  Further,  the  implied  mathematical  rigor  would  detract  rather  than 
enhance  the  credibility  and  net  worth  of  the  effort.  Thus,  we  propose  to  allocate  weighing 
factors  to  the  system  characteristics  listed  above  using  judgement  and  an  assessment  of  the 
desires  of  the  ultimate  user  - the  DOD  and  other  government  agencies. 

The  following  series  of  quotations  from  statements  made  by  government  leaders  in 
discussions  of  the  DCS,  WWMCCS,  and  the  interplay  between  the  two  should  bear  heavily  upon 
the  judgement  applied  to  this  portion  of  the  program.  All  of  the  following  quotations  are  extracted 
from  the  March  1975  issue  of  Signal  Magazine,  a special  issue  on  Command  and  Control. 

Secretary  of  Defense,  James  R.  Schlesigner  "Our  National  Security  strategy 
places  emphasis  on  providing  the  President  with  greater  flexibility  and  selectivity  in  the  nuclear 
response  options  available  to  him  in  times  of  crisis.  Implicit  in  this  strategy  is  the  need  for 
a responsive,  flexible  and  survivable  WWMCCS  that  provides  appropriate  command  and  control 
not  only  for  the  strategic  forces  but  also  for  the  tactical  forces  as  well . " 
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Thomas  C.  Reed,  Director,  Telecommunications  and  Command  and  Control 

3 

Systems,  OSD.  "Implicit  to  this  objective  is  the  recognition  that  C is  integral  to  the 
full  structure  and  deployment;  that  survivability  of  the  functional  command  and  control 
compatibility  is  fundamental  to  continuity  of  operations  under  stress  conditions;  and  that  the 

3 

C elements  must  be  sufficiently  dynamic  to  adapt  to  the  rapidly  changing  environment." 

Lt.  Gen.  Lee  M.  Paschall,  USAF,  Commander  Defense  Communications  Agency. 
"The  connectivity  provided  by  large  scale,  general  purpose  communication  network  such  as 
the  DCS,  is  essential  to  the  operation  of  a worldwide  command  and  control  system.  The  DCS 
with  its  more  than  42M  channel  miles  of  various  types  of  circuits  interconnecting  more  than 
1 ,000  locations  throughout  the  world  provides  the  majority  ofall  WWMCCS  communications 
requirements"  - General  Paschal  I continues , "It  ensures  the  flexibi  lity,  survivability, 
redundancy,  and  finally,  the  restorability  the  communications  network  supporting  command 
and  control  requirements  must  have."  - Also,  General  Paschall  states:  "The  priorities 
under  which  the  DCS  is  engineered,  operated,  and  managed  state  with  unmistakable  clarity 
that  communications  which  support  command  and  control  are  examined  and  weighed  on  the 
basis  of  their  survivability,  reliability,  security  and  cost  - and  in  that  order."  - It  is 
difficult  to  predict  precisely  at  this  time  the  communications  requirements  for  interconnecting 
the  WWMCCS  and  the  various  tactical  command  and  control  systems.  However,  it  seems 
certain  that  the  future  DCS  will  be  heavily  oriented  towards  interoperability  with  U.S . 
tactical  communications  systems  and  with  the  communication  systems  of  our  major  allies.  - 
The  DCS  quite  obviously  cannot  satisfy  all  the  command  and  control  requirements  inherent 
in  flexible  response  strategy  and  scene  of  action  force  control.  However,  it  is  the  only 
system  capable  of  interconnecting  the  National  Command  Authority  with  the  communications 
serving  the  operating  forces." 

General  Russell  E.  Dority,  USAF,  Commander  in  Chief  of  the  Strategic  Air 
Command.  "Once  a decision  is  made  to  protect  or  employ  nuclear  forces,  the  combat 
crews  must  receive  the  validated  directives  with  a degree  of  accuracy  and  reliability  that 
cannot  be  compromised  or  confused.  There  must  be  no  opportunity  for  misrepresentation  of  valid 
directives  - nor  possibility  of  improper  reaction  to  unauthorized  orders  from  any  source"  - 
"The  attributes  discussed  earlier  characterize  the  resources  of  quality  that  must  be  realized  in 
the  system;  reliability,  survivabi  lity,  flexibility,  speed,  and  accuracy.  Without  reliability. 
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other  system  attributes  have  little  meaning.  Reliability  is  a prime  concern  of  the  SAC  , j 

control  system.  Survivability  is  also  a crucial  objective  for  any  military  communication 

system,  and  its  importance  to  SAC  is  self-evident.  Flexibility  implies  a capability  to  accommodate 

and  adapt  to  the  full  range  of  changing  environments  and  unplanned  requirements." 

Maj.  Gen.  Herbert  Summers  Cunningham,  USA,  Chairman  WWMCCS  Council 
Support  Group.  "The  basic  qualities  required  of  such  a system  (WWMCCS)  were  credibi lity, 
flexibility,  responsiveness,  survivability  and  security.  These  qualities  must  exist  to  the  maximum 
extent  possible  within  the  restraints  imposed  by  resources  and  technology." 

Maj.  Gen.  Thomas  M.  Rienzi,  USA  - Director  of  Telecommunications  and  Command 
and  Control,  DCSO,  U.S.  Army.  "Regardless  of  the  size  of  the  command  and  control  system 
or  the  echelon  it  supports,  such  a system  must  be: 

• Survivable  - At  least  to  the  extent  that  the  people  it  supports  are  survivable. 

• Integrated  - With  other  administrative  and  management  systems  outside  the 
organization  chain  of  command  to  permit  collection,  processing  and  transfer 
of  information  and  decisions  responsibility  without  unduly  burdening  the 
tactical  commanders  for  the  reports  or  reporting. 

• Responsive  - To  provide  rapid  feedback  to  the  orginal  source  of  information 
to  confirm  accuracy  of  perception  by  the  recipient. 

• Flexible  - To  provide  for  the  coordination  and  direction  required  to  change 
missions  or  resources  allocations." 

In  light  of  the  above  and  with  the  goal  of  not  complicating  this  process,  it 
has  been  decided  to  divide  the  characteristics  into  a primary  list  and  a secondary  list.  Those 
in  the  primary  list  will  have  greater  weighing  than  those  in  the  secondary  list.  The  primary 
list  consists  of  those  characteristics  that  directly  relate  to  the  characteristics  mentioned  in 
the  above  quotations.  Those  characteristics  that  do  not  directly,  but  rather  indirectly  provide 
the  key  characterisitics,  have  been  placed  in  the  secondary  list. 


9-7 


Primary  Characteristics 


r 


Survivability 
Susceptibility 
Flexibility 
Interoperability 
Availabi  lity 

Secondary  Characteristics 
Error  Propagation 

Compatibility  with  Other  Global  Timing  Sources 

Precise  Time  Availability 

Stability 

Maintainability 


SECTION  10.0 
COST  EVALUATION 
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The  analysis  carried  out  in  the  costing  area  was  not  an  absolute  costing  of  each 
technique.  Rather,  what  was  attempted  was  to  determine  if  any  of  the  techniques  was  inherently 
more  expensive  irrespective  of  the  implementation  of  the  technique. 

In  comparing  costs  between  the  candidates  there  are  some  significant  points  to 
be  considered  at  the  outset.  First,  timing  subsystem  costs  are  dominated  by  clock  costs. 

An  ROM  cost  for  a cesium  clock  is  $25,000.  An  ROM  cost  for  a good  crystal 
clock  is  $3500.  The  remaining  hardware  that  comprises  the  timing  subsystem,  irrespective  of 
technique  and  as  shown  in  Figures  10.0-1 , 10.0-2,  10.0-3,  and  10.0-4,  has  an  ROM  cost  of  less 
than  $500. 

Second,  only  the  independent  clocks  technique  must  be  configured  with  expensive 
cesium  clocks  at  every  node.  The  disciplined  techniques  can  utilize  a majority  of  less  expensive 
crystal  clocks  with  the  placement  of  the  cesium  clocks  determined  by  the  network  connectivity 
pattern. 

Therefore,  the  conclusion  has  been  reached  that  cost  differentials  between  the 
disciplined  techniques  are  not  significant.  Also,  that  the  independent  clocks  approach  is 
inherently  more  expensive. 

10.1  Relative  Costs 

Relative  costs  can  be  explained  most  easily  by  reference  to  Figures  10.0-1 
through  10.0-4.  Figure  10.0-1  is  a simplified  functional  block  diagram  of  a node  in  a 
network  utilizing  an  independent  clock  approach. 

The  row  of  components  along  the  top  shows  the  flow  of  data  traffic  in  the  node 
and  is  the  same  for  all  the  techniques.  Each  demodulator  at  a node  (the  i^  demodulator  is 
illustrated)  provides  an  input  to  a demultiplexer.  The  demultiplexer  provides  a number  of 
lower  bit  rate  outputs,  each  of  which  is  written  into  an  elastic  stores  buffer  at  a rate  determined 
from  the  bit  sync  in  the  demodulator.  Data  is  read  out  of  every  buffer  at  the  node  at  a rate 
determined  by  the  nodal  clock  and  all  such  data  streams  are  therefore  synchronous.  The  nodal 
switch  and  multiplexers  are  all  timed  by  the  same  nodal  clock. 


Figure  10.0-2.  Master-Slave  Functional  Block  Diagram 


RECEIVE  TIMING 


Figure  10.0-3.  Mutual  Synchronization  Functional  Block  Diagram 


Figure  10.0-4.  TRD  Functional  Block  Diagram 


In  the  independent  clock  approach,  the  free  running  clock  provides  this  timing  to 
the  buffer  outputs,  the  switch  and  the  multiplexers.  As  stated  earlier,  this  clock  is  undoubtedly 
a cesium  standard  and  would,  therefore,  have  an  ROM  acquisition  cost  of  $25,000. 

Figure  10.0-2  is  a simplified  functional  block  diagram  of  a master-slave  node. 
Again,  the  same  components  are  shown  along  the  top  row.  Timing  as  derived  in  each  of  the 
demodulator  bit  syncs  is  applied  to  a link  selection  switch  which  chooses  the  appropriate 
reference  to  apply  to  the  phase-lock  loop.  The  loop  locks  the  nodal  clock  to  the  receive  timing 
signal.  As  mentioned  earlier,  the  nodal  clock  need  not  be  a cesium  standard  since  it  is  a 
disciplined  clock.  Assuming  10  percent  of  the  clocks  in  the  network  are  cesium  clocks  with  a 
$25,000  cost  and  the  remaining  clocks  are  crystal  clocks  with  a $3500  cost,  the  average 
nodal  clock  cost  is  $5650.  The  switch,  phase  detector  and  loop  filter  that  comprise  the  remain- 
der of  the  nodal  timing  subsystem  can  be  implemented  for  an  ROM  cost  of  $300. 

Figure  10.0-3  depicts  a simplified  functional  block  diagram  of  a mutual  synchron- 
ization node.  Again,  receive  timing  is  derived  from  each  of  the  modem  bit  syncs.  These  clocks 
are  then  applied  to  the  phase  defector  in  the  loop  shown.  Phase  errors  for  each  of  the  inputs 
are  weighted  and  summed  and  then  applied  to  the  loop  filter.  In  turn,  the  output  of  the  filter 
provides  the  discipline  to  the  local  nodal  clock. 

The  nodal  clock  need  not  be  a cesium  standard.  If  the  10  percent  assumption  is 
made  as  before,  the  average  clock  cost  is  $5650. 

The  phase  detector,  weight  and  sum  circuitry  and  loop  filter  could  be  implemented 
for  an  ROM  cost  of  $300. 

Figure  10.0-4  depicts  a simplified  functional  block  diagram  of  a TRD  node.  The 
same  components  shown  in  the  previous  figures  are  along  the  top  row.  The  components  noted 
along  the  second  row  form  the  heart  of  the  implementation  difference  between. the  TRD  approach 
and  the  other  disciplined  techniques.  The  nodal  clock  shown  below  is  present  in  all  the 
techniques.  The  nodal  frame  generator  shown  at  the  lower  right  is  also  present  independent 


The  demux  provides  inputs  to  each  of  the  three  components  across  the  center  of 
the  figure.  The  link  selection  processor  on  the  left  receives  node  and  link  status  information. 

It  is  the  basis  for  the  adaptive  reorganization  feature.  It  could  be  implemented  with  one 
microprocessor  chip,  one  interface  device  chip,  five  1024-word  Read  Only  Memories,  and 
a 1024-word  Random  Access  Memory.  An  ROM  cost  would  be  under  $200. 

The  start/stop  counter  measures  the  time  delay  between  the  transmitted  framing 
bit  and  the  receive  framing  bit.  This  measurement  is  required  in  the  time  transfer  calculation 
to  be  made  by  the  clock  error  processor.  A binary  counter  essentially  provides  this  function. 

An  ROM  cost  would  be  $100. 

The  clock  error  processor  on  the  right  receives  from  the  demultiplexer  the  delays 
measured  at  the  adjacent  nodes  which  is  the  remaining  information  necessary  to  make  the  time 
transfer  calculation.  Based  on  this  calculation  and  the  clock  offsets  at  the  adjacent  nodes, 
also  provided  by  the  demux,  a local  correction  is  generated.  It  would  appear  that  the  clock 
error  processing  can  be  done  by  the  same  components  that  constitute  the  link  selection  processor. 

As  in  the  other  disciplined  approaches  the  average  clock  cost  is  estimated  to  be 
$5650  and  the  entire  nodal  timing  subsystem  to  be  $5950. 

In  summary,  acquisition  costs  for  each  of  the  disciplined  techniques  appear  equal. 
The  independent  clock  approach  has  higher  acquisition  costs. 

Nothing  was  found  to  indicate  that  maintenance  costs  would  be  significantly 
different  from  technique  to  technique. 

Likewise,  there  is  little  to  indicate  that  front-end  development  costs  would  be 
significantly  different  for  any  technique.  The  master-slave  technique  has  been  implemented 
by  Bell  and  others.  The  TRD  approach  has  been  essentially  implemented  in  the  Canadian  Data 
Route.  The  independent  clock  technique  is  being  developed  by  Tri-Tac.  The  mutual  sync 
technique,  while  not  implemented  to  our  knowledge,  has  certainly  been  exhaustively  analyzed 
in  the  literature. 
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alternative  timing  subsystems 


Introduction 


Each  of  four  techniques  have  been  extensively  examined  with  respect  to  the 
feasibility  of  their  application  to  the  DCS  timing  subsystem.  A significant  conclusion  that  was 
anticipated  earlier  and  confirmed  during  this  work  was  that  limiting  the  timing  subsystem  to  the 
use  of  a single  technique  would  be  overly  restrictive.  Network  configurations  and  the  inherent 
characteristics  of  each  technique  guarantee  that  such  a restriction  will  only  provide  a suboptimal 
solution.  Discussed  below  are  candidate  timing  subsystems  that  are  both  feasible  and  not  limited 
by  such  a restriction.  Preceding  this,  the  key  factors  that  impacted  the  selection  of  the  candi- 
date timing  sybsystems  are  discussed.  It  should  be  noted  that  the  results  of  Section  8.0  were  not 
used  to  eliminate  candidate  timing  subsystems  as  such;  rather  the  results  of  Sections  8.0,  9.0 
and  10.0  will  be  used  in  the  concluding  section  to  select  the  best  of  the  candidates. 


i 1.2  Network  Considerations 

The  network  configuration  influences  what  combinations  of  timing  techniques 
form  an  appropriate  timing  subsystem.  In  turn,  the  requirements  on  timing  components  are 
surely  dependent  on  timing  technique.  However,  due  to  the  relative  small  cost  of  the  timing 
subsystem  in  relation  to  the  overall  cost  of  the  DCS  network,  it  is  not  feasible  that  timing 
considerations  should  dictate  any  overriding  limitations  on  the  present  or  future  characteristics 
of  the  network  itself.  Thus,  the  timing  subsystem  must  be  made  to  fit  the  requirements  of  the 
network  and  not  vice  versa. 

The  network  may  be  thought  of  as  being  composed  of  a hierarchy  of  nodes  with 
different  numbers  of  nodes  at  each  level  of  the  hierarchy  and  a certain  amount  of  connectivity 
between  the  nodes  at  each  level  as  well  as  some  connectivity  between  levels. 

The  key  factor  which  determines  the  level  to  which  a node  belongs  is  the  critical- 
ity or  importance  of  the  node.  In  general,  this  factor  varies  with: 

• Amount  of  traffic  handled 

• Geographic  location  of  node 

• Number  of  other  nodes  directly  connected  to  it 


On  the  basis  of  these  factors,  a number  of  levels  could  be  postulated  for  the  DCS 
network,  but  it  is  believed  that  three  levels  is  adequate  for  representing  its  essential  character- 
istics. At  each  level  there  exists  three  basic  connection  types.  These  types  are:  1)  fully 
connected;  2)  ring  connection;  and  3)  star  connection.  The  topologies  of  these  basic  connection 
types  are  shown  in  Figure  11.2-1.  The  actual  network  connectivity  will  very  likely  not  fit  any 


Ring  Connection 


Figure  11.2-1.  Three  Basic  Network  Connection  Types 
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of  the  three  basic  types  at  any  of  its  hierarchical  levels.  At  the  highest  level,  the  actual 
network  may  be  characterized  as  consisting  of  several  highly  connected  clusters  with  a small 
amount  of  connectivity  between  clusters.  An  exaggerated  form  of  this  type  of  connectivity  is 
shown  in  Figure  11.2-2.  Even  though  the  highest  level  of  the  actual  DCS  network  may  not  be 
fully  connected,  it  is  sufficiently  connected  that  any  of  the  four  generic  timing  subsystem 
configurations  may  be  applied  at  this  level. 


Figure  11.2-2.  Clustering  Network  Connectivity 
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Intermediate  level  nodes  tend  to  be  clustered  around  the  highest  level  nodes.  In 
each  cluster  of  intermedial  level  nodes,  the  total  connectivity  may  vary  considerably  from  one 
cluster  to  the  next,  but  the  connectivity  between  intermediate  level  clusters  is  small. 

The  lowest  level  nodes  tend  to  be  connected  directly  to  only  a single  intermediate 
or  highest  level  node  with  little  connection  between  other  nodes  at  the  lowest  level. 

11.3  Major  Network  Synchronization  Components 

The  major  network  synchronization  components  include  the  following: 

• Stable  clocks 

• Data  buffers  at  each  node 

• Phase  lock  loops 

• Transmission  media 

• Fault  detection  and  recovery  protocol 

• Maintenance  aids 

The  various  timing  subsystem  techniques  place  different  requirements  on  these 
components,  individually  and  collectively.  Distinct  possibilities  exist  for  trading  severity  of 
requirements  on  one  component  for  those  on  another  in  accordance  with  the  timing  subsystem 
that  is  chosen  for  the  DCS.  Requirements  on  these  components  may  also  vary  in  time  as  the 
DCS  is  utilized  in  new  applications. 

11.4  Timing  Subsystems 

From  the  preceding  discussion,  it  is  apparent  that  any  one  of  the  four  generic 
timing  subsystem  techniques  is  applicable  at  the  highest  level  of  the  DCS  hierarchy.  It  also 
seems  that  due  to  connectivity  of  the  network  that  the  lowest  level  nodes  should  be  slaved  to 
their  corresponding  intermediate  or  highest  level  node.  This  results  from  the  limited  connectiv- 
ity that  characterizes  the  lowest  level  nodes.  Thus,  we  are  left  with  deciding  which  generic 
techniques  at  the  intermediate  level  are  most  appropriate  given  that  a particular  technique  has 


been  chosen  for  the  highest  level  of  the  network.  Some  pairings  of  techniques  at  the  highest 
and  intermediate  levels  are  illogical  and  consequently  should  not  receive  serious  consideration. 
For  example,  non-time  reference  distribution  at  the  highest  level  nodes  and  time  reference 
distribution  at  the  intermediate  level  is  unsuitable  because  the  greatest  time  and  phase 
accuracies  should  be  available  at  the  major  nodes  and  timing  information  dissemination  should 
flow  from  the  higher  to  the  lower  levels  in  accordance  with  the  criterion  for  ranking  the  nodes 
of  the  network.  Another  factor  which  should  be  kept  in  mind  is  that  there  exists  a natural 
tendency  to  slave  lower  level  nodes  to  higher  level  nodes  via  highest  quality  links.  Due  to  the 
topology  at  the  intermediate  level,  this  in  general  translates  into  slaving  a cluster  of  interme- 
diate nodes  to  the  cluster's  primary  higher  level  node  via  a highest  quality  link.  Fallback 
protocol  for  first  alternate  would  then  involve  switching  to  alternate  links  and  second  alternates 
would  involve  slaving  to  alternate  higher  level  nodes.  Thus,  we  see  that  a natural  tendency  is 
against  progression  from  tighter  coupling  to  looser  coupling  when  traversing  from  the  higher 
level  nodes  to  the  lower  level  nodes.  It  then  follows  that  given  mutual  sync  or  master-slave 
at  the  highest  level,  mutual  sync  or  master-slave  at  the  intermediate  level  is  most  appropriate. 

With  these  tendencies  in  mind,  the  following  timing  subsystems  seem  good  candi- 
dates for  analysis  and  simulation: 

1.  Master-Slave  Throughout 

A further  impetus  for  including  this  timing  subsystem  is  that  almost  all 
commercial  systems  will  utilize  this  form  of  timing  subsystem. 

2.  Mutual  sync  at  highest  and  intermediate  levels  with  lowest  level  nodes 
slaved  to  their  corresponding  intermediate  or  highest  level  node. 

3.  Time  reference  distribution  at  highest  level,  master-slave  at  intermediate 
level,  and  each  lowest  level  node  slaved  to  corresponding  intermediate  or 
highest  level  node. 

This  timing  subsystem  will  combine  the  attributes  of  time  reference  distri- 
bution with  the  simplicity  of  the  master-slave  technique  applied  to  the 
majority  of  the  nodes  in  the  network. 


4.  Time  reference  distribution  at  highest  level,  mutual  sync  at  intermediate 
level  and  each  lowest  level  node  slaved  to  corresponding  intermediate  or 
highest  level  node. 

This  timing  subsystem  should  have  greater  potential  for  survival  than  (3)  but 
will  retain  the  attributes  of  the  TRD  configuration. 

5.  Time  reference  distribution  at  both  the  highest  level  nodes  and  the  interme- 
diate level  nodes  and  each  lowest  level  node  slaved  to  one  of  the  interme- 
diate or  highest  level  nodes. 

6.  Independent  clock  at  highest  level,  master-slave  at  intermediate  level,  and 
lowest  level  nodes  slaved  to  either  an  intermediate  or  a highest  level  node. 
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CONCLUSIONS 


The  results  of  both  the  analysis  and  simulation  efforts  have  led  to  the  recommenda- 
tion of  time  reference  distribution  as  the  technique  to  employ  at  the  major  nodes  of  the  future 
switched  digital  Defense  Communications  System.  The  analysis  effort  indicated  that  as  a tech- 
nique, TRD  ranked  higher  in  more  of  the  primary  as  well  as  more  of  the  secondary  desirable 
characteristics  than  any  of  the  other  techniques.  The  simulation  indicated  that  TRD  could  main- 
tain closer  frequency  coordination  between  nodal  clocks  in  the  face  of  both  stress  and  normal 
operating  conditions.  The  cost  analysis  indicated  that  the  performance  benefits  resulted  without 
an  increase  in  the  cost  of  the  nodal  timing  subsystem  equipment. 

In  examining  the  results  of  this  study,  the  high  evaluation  of  time  reference 
distribution  resulted  from  four  key  features  of  the  technique.  The  most  significant  feature  was 
the  double  endedness  of  the  approach.  This  removed  the  effect  on  nodal  clocks  caused  by 
variations  in  link  phase  delay.  The  second  significant  feature  was  the  adaptive  reorganization 
capability  which  assured  the  utilization  of  the  best  possible  available  reference  after  a failure. 
This  feature  resulted  in  smaller  differences  between  the  instantaneous  frequencies  of  the  nodal 
clocks  after  such  failures.  The  third  feature  was  the  natural  separation  of  error  measurement  and 
error  correction.  This  feature  which  allows  simultaneous  correction  of  each  clock  with  respect 
to  the  master  clock  rather  than  the  disciplining  clock,  also  results  in  minimizing  the  instantaneous 
frequency  differences  between  nodal  clocks.  The  fourth  significant  characteristic  was  the  provi- 
sion of  precise  time  which  opens  the  door  for  many  potential  benefits  as  well  as  aiding  moni- 
torability,  and  helping  to  ensure  synchronous  operation. 

In  summarizing  the  capabilities  of  the  techniques  it  is  worthwhile  to  reiterate  a 
point  made  in  Section  2.0  of  this  report:  Any  of  the  techniques,  time  reference  distribution, 
master-slave,  mutual  synchronization  or  independent  clocks  can  be  made  to  work.  The  differences 
in  performance  capabilities  between  the  techniques  while  clear,  do  not  preclude  the  use  of  any 
of  the  techniques.  Furthermore  some  of  the  four  key  features  of  time  reference  distribution  dis- 
cussed above  can  in  certain  instances  be  included  in  a few  of  the  other  techniques.  To  the  degree 
that  they  are,  the  differences  in  capabilities  between  the  techniques  narrows.  The  approach  taken 
in  this  study  was  to  lump  the  four  features  strictly  into  the  time  reference  distribution  approach. 

The  rationale  was  that  each  of  these  features  depends  upon  an  overhead  link  capability  for  its 
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operation.  Once  the  overhead  link  is  provided,  one  is  essentially  settling  for  an  inferior  subsystem 
if  all  four  of  the  features  are  not  included;  and  TRD  is  the  only  technique  that  can  provide  all  the 
features. 

Having  chosen  time  reference  distribution  as  the  timing  technique  to  be  utilized 
at  the  major  DCS  nodes,  it  was  clear  that  master-slave  should  be  implemented  at  the  lower  level 
nodes.  (See  Section  11  .0).  At  the  lower  level  nodes  in  the  network  which  are  characterized  by 
limited  connectivity  and  lower  bit  rates  the  benefits  of  TRD  that  are  highlighted  at  a highly  con- 
nected node  are  not  as  significant.  Also  the  overhead  channel  required  for  TRD  would  represent 
a larger  percentage  of  the  data  traffic. 

Although  in  Section  11 .0  two  such  candidates  were  discussed,  differing  only  in  the 
technique  to  be  used  in  the  intermediate  nodes,  one  is  reluctant  to  choose  between  the  two.  This 
is  because  of  the  somewhat  loose  and  arbitrary  designation  of  three  nodal  levels.  Suffice  it  to  say 
that  the  results  of  this  study  should  make  one  very  comfortable  with  1)  the  selection  of  time 
reference  distribution  at  major  nodes  because  of  its  overall  utility  and  2)  the  selection  of  master- 
slave  at  the  lower  level  nodes  where  many  of  the  TRD  benefits  are  precluded  by  network 
configurations. 
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13.0 


TESTING  PROGRAM 


The  conclusions  reached  in  this  study  were  based  on  substantial  analytical  and 
computer  simulation  efforts.  It  is  reasonable  to  expect  that  a well  organized  experimental 
test  effort  will  provide  additional  data.  Outlined  below  is  such  a testing  program.  It  is 
suggested  that  media  and  transmission  rates  included  in  the  test  program  should  be  troposcatter 
to  13  Mb/s,  microwave  LDS  to  20  Mb/s,  and  satellite  to  180  Mb/s. 

13.1  Parameters  To  Be  Tested 

The  following  paragraphs  describe  the  appropriate  parameters  to  be  tested. 

Phase  Jitter 

Jitter  or  instability  of  the  phase  of  the  bit  synchronizer  timing  waveform  is  an 
important  parameter.  The  spectrum  of  this  jitter  should  be  determined  under  a representative 
set  of  conditions;  different  types  of  links,  data  rates,  signal -to-noise  rations,  jamming  environ- 
ments, etc.  Jitter  is  an  important  parameter,  affecting  bit  error  rate  and  the  probability  of  a 
bit  slip  in  the  bit  sync.  It  also  affects  nodal  synchronizer  clock  stability  if  that  clock  is  derived 
from  the  bit  sync  timing  waveform,  such  as  in  the  master-slave  and  mutual  sync  techniques.  This 
parameter  impacts  a number  of  characteristics  by  which  timing  techniques  were  evaluated, 
including  bit  count  integrity,  bit  and  frame  sync  performance,  susceptibility  to  jamming,  surviv- 
ability, error  propagation,  and  precise  time  availability.  In  most  of  these  cases,  the  impact 
is  on  slip  rate  which,  in  turn,  impacts  quality  of  digital  communication  in  a digital  network. 

For  the  error  propagation  characteristic,  knowledge  of  the  phase  jitter  spectrum  allows  selecting 
a design  such  that  error  propagation  is  not  troublesome.  (For  example:  The  proper  loop  band- 
width in  a master-slave  timing  subsystem  can  be  chosen.)  For  the  precise  time  availability 
characteristic,  phase  jitter  characteristics  must  be  known  to  determine  the  accuracy  of  time 
transfer  techniques  such  as  are  used  in  the  time  reference  distribution  approach. 

Propagation  Delays 

Several  measurements  should  be  carried  out  in  this  area  including  total  propaga- 
tion and  equipment  delays  between  two  nodes,  the  variation  in  this  total  delay  as  a function  of 
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multipath  or  other  effects,  and  difference  in  time  delay  between  the  paths  in  both  directions 
between  two  nodes  (reciprocal  path  delays).  These  parameters  cause  several  effects.  Variations 
in  path  delay  result  in  additional  phase  jitter  on  the  receive  timing  waveform  (and  the  effects 
caused  by  phase  jitter  as  discussed  above)  and  cause  a requirement  for  additional  buffering  at 
communication  nodes  to  smooth  out  these  variations.  Differences  in  reciprocal  path  delays 
impose  a limit  on  the  accuracy  with  which  time  transfers  can  be  accomplished  and,  therefore, 
a limit  on  the  accuracy  of  the  time  reference  distribution  technique.  These  parameters  impact 
a number  of  characteristics,  including  bit  count  integrity,  bit  and  frame  sync  performance, 
survivability,  error  propagation,  and  precise  time  availability. 

RSL,  S/N,  and  C/N 

The  quantities  received  signal  level  (RSL),  signal -to-noise  ratio  (S/N),  and 
carrier-to-noise  ratio  (C/N)  are  indicative  of  channel  conditions.  These  quantities  should  be 
measured  and  correlated  with  other  quantities,  such  as  bit  error  rate,  fade  rate/duration/depth, 
and  bit  count  integrity  for  each  type  of  link.  Certain  quantities,  such  as  very  small  values  of 
bit  slip  probabilities,  should  probably  be  measured  only  once  on  each  link  as  a function  of  S/N. 
The  quantities  RSL,  S/N,  and  C/N  do  not  all  provide  equivalent  information.  One  could  have 
both  a high  RSL  and  a low  S/N  if,  for  example,  one  were  in  a jamming  environment.  These 
quantities  are  not  only  highly  correlated  with  other  parameters  that  should  be  measured,  but 
they  impact  several  key  timing  subsystem  characteristics.  These  characteristics  are  bit  count 
integrity,  data  rates,  bit  error  rates/signal  outages/fading,  bit  and  frame  sync  performance,  and 
susceptibility  to  jamming. 

Bit  Error  Rates  at  Various  Transmission  Rotes 

Link  measurements  should  be  made  to  determine  bit  error  rate  for  various  trans- 
mission rates  on  each  link  as  a function  of  RSL,  S/N,  and  C/N.  Thus,  bit  error  rate  can  be 
determined  for  a variety  of  channel  conditions  such  as  fading,  jamming,  etc.  Average  bit 
error  rate  is  one  of  the  fundamental  parameters  that  determines  the  quality  of  digital  communi- 
cation in  a digital  network.  In  addition  to  the  average  error  rate,  statistics  of  the  error  events 
should  be  determined.  This  parameter  impacts  several  characteristics,  including  bit  count 
integrity,  susceptibility  to  error  rate,  bit  and  frame  sync  performance,  and  signal  outages/fading. 
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Bit  errors  can  affect  the  nodal  synchronizer  since  they  can  corrupt  control  information  sent 
from  node  to  node.  This  can  occur  with  any  technique  that  requires  control  information,  such 
as  master-slave,  time  reference  distribution,  or  pulse  stuffing. 

Bit  Count  Integrity 

Bit  count  integrity  is  necessary  for  maintaining  frame  sync  for  multiplexer,  switches 
and  crypto  sync.  This  parameter  should  be  measured  by  determining  the  bit  sync  slip  rates  as  a 
function  of  RSL,  S/N,  and  C/N  for  a variety  of  links  and  data  rates.  Characteristics  affected 
by  this  parameter  are  frame  sync  performance,  communications  security,  survivability,  and 
susceptibility  to  jamming. 

Fade  Rate/Duration/Depth 

Fading  characteristics  of  each  type  of  link  will  be  obtained  as  a function  of 
frequency,  data  rate,  and  distance.  These  characteristics  should  be  correlated  with  RSL,  S/N 
and  C/N.  In  fact,  RSL,  S/N,  and  C/N  provide  a means  by  which  these  fading  characteristics 
may  be  measured.  The  fade  rate  indicates  the  rapidity  with  which  a fade  occurs,  i.e.,  the 
signal  level  rate  of  decrease  is  measured  in  dB/sec.  Knowledge  of  this  quantity  is  important 
in  implementing  hardware  for  detecting  the  onset  of  fades  and  providing  switchover  to  diverse 
paths  not  experiencing  a fade.  Fade  duration  is  the  length  of  an  outage  (in  seconds),  and  fade 
depth  is  the  amount  by  which  the  signal  level  has  decreased  (in  dB)  from  its  nominal  value  (with 
no  fading).  Since  each  fade  is  different,  these  parameters  should  be  characterized  statistically, 
i.e.,  a probability  distribution  of  fade  rates,  fade  durations,  and  fade  depths  will  be  obtained 
for  each  case.  Characteristics  impacted  by  the  fading  parameters  are  bit  count  integrity,  bit 
and  frame  sync  performance,  error  propagation,  and  susceptibility  to  error  rate.  Any  network 
timing  technique  which  attemps  to  derive  the  nodal  clock  from  a fading  signal  will  be  affected 
to  some  degree  by  this  phenomena.  About  the  only  approach  not  affected  is  independent  clocks. 

Buffer  Fill 


This  is  an  example  of  a quantity  that  would  be  monitored  in  a network  test. 
Buffer  fill  provides  a measure  of  the  absolute  phase  difference  between  the  nodal  clock  and 
the  receive  timing  waveform.  Buffer  overflows  can  be  a significant  source  of  bit  slips.  Thus, 
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the  buffer  size  required  to  make  the  effect  of  these  bit  slips  on  system  performance  negligible  is 
a significant  design  parameter  for  all  timing  techniques.  This  requirement  will  vary,  depending 
on  the  type  of  link  and  data  rate.  This  parameter  will  impact  several  characteristics,  including 
bit  count  integrity,  frame  sync  performance,  susceptibility  to  jamming,  and  survivabil ity . 

The  parameters  described  above  should  be  measured  in  several  different  types  of 
tests  including  point-to-point,  tandem  link,  and  network.  The  number  of  parameters  measured 
in  each  type  of  test  should  get  successively  smaller.  Parameters  to  be  measured  on  point-to- 
point  tests  involve  a variety  of  link  types  and  all  parameters  discussed  above,  including  phase 
jitter,  propagation  delay  measurements,  bit  count  integrity,  bit  error  rates  at  various  trans- 
mission rates,  fade  rate/duration/depth,  and  RSL,  S/N,  C/N.  In  proceeding  to  tandem  link 
tests,  no  additional  information  is  provided  by  making  further  measurements  of  fade  rate/duration/ 
depth  or  RSL,  S/N,  C/N.  However,  statistics  of  several  parameters  change  in  passing  through 
several  tandem  links,  and  measurements  of  these  effects  must  be  made  to  accurately  model  them. 
The  parameters  measured  in  tandem  link  tests  should  be  accumulated  phase  jitter,  propagation 
delay  measurements,  bit  error  rate,  and  bit  count  integrity.  Measurements  should  be  made 
for  simple  IF  repeaters  and  regenerative  repeaters  that  retime  the  transmitted  digital  sequence. 

All  tests  done  to  this  point  involve  gathering  data  that  can  be  used  to  improve  the  accuracy 
and  validity  of  the  analytical  work.  Network  tests  involve  an  evaluation  of  a specific  network 
synchronization  approach.  In  performing  this  evaluation,  buffers  at  each  node  to  temporarily 
store  the  received  data,  and  a nodal  synchronizer  of  some  type  to  retime  the  digital  sequences 
at  each  node  are  required.  Several  network  configurations  representative  of  the  future  DCS 
can  be  used.  Basic  parameters  to  be  monitored  in  this  type  of  test  are  buffer  fill  at  each  node 
and  bit  slips  due  to  both  slips  in  bit  syncs  and  buffer  overflows  (for  various  size  buffers).  These 
parameters  should  be  monitored  for  various  network  configurations,  fading  environments,  and 
stress  scenarios.  A major  result  of  this  series  of  tests  should  be  a verification  of  the  analytical 
and  simulation  results  obtained  thus  far. 

13.2  Summary  and  Test  Example 

A general  listing  of  tests  includes  media  from  Table  13.2-1  and  parameters  from 
Table  13.2-2.  Each  test  should  likely  be  run  as  functions  of  frequency  and  data  rates.  Efforts 
should  be  made  to  require  testing  to  be  applied  to  equipment  to  be  used  in  the  timing  system. 
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Table  13.2-1.  Communications  Media 


Ground-to-satellite  links 


Microwave  links 


Troposcatter  radio  links 
Wire  line  links 


HF  radio  links 


Table  13.2-2.  Critical  Operational  Parameters 
Phase  jitter 
Doppler  effects 
Propagation  delays 
Differential  propagation  delays 
Reciprocal  propagation  delays 
Received  signal  level 
Signal -to-noise  ratio 


Carrier-to-noise  ratio 


Bit  error  rate 


Bit  count  integrity 
Bit  synchronism 


Fade  rate 


Fade  duration 


Fade  depth 


Buffer  fills 


Synchronism  losses  (frame  and  crypto) 


Time  to  establish  synchronism  (initial  and  resync) 
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Test  Example 


This  paragraph  is  an  example  test  plan  included  to  show  how  a typical  test  should 
be  documented. 

The  test  example  is  actually  two  separate  tests  that  establishes  a baseline  for  a 
third  test  which  is  of  prime  importance  to  the  Time  Reference  Distribution  scheme  for  network 
timing.  Ultimately  this  test  example  should  be  worked  into  three  separate  subplans  which  will 
define  the  tests  in  more  detail  than  presented  here.  Because  of  the  similarity  of  the  tests,  all 
three  are  discussed  here.  These  tests  are:  1)  measurement  of  propagation  delay,  2)  measurement 
of  differential  time  delays  and  3)  measurement  of  the  reciprocal  time  delays  (differential  delay 
between  both  directions  of  a full  duplex  link).  This  example  considers  these  delays  as  associated 
with  a troposcatter  radio  link. 

a.  Test  Objective  - To  determine  the  absolute  time  delay  between  two  tropo- 
scatter radio  terminals  and  the  frequency  dependent  differential  delays.  This 
measurement  includes  both  the  propagation  delay  and  equipment  delay. 
Corollary  tests  should  be  defined  to  determine  equipment  delays  and  recip- 
rocal time  delays. 

b.  Definition  of  Parameters  to  be  Measured 

Propagation  Delay  - The  total  time  differential  in  nanoseconds  required  for 
the  coding,  modulation,  transmission,  propagation,  reception,  demodulation 
and  decoding  of  a 13  Mb/s  data  stream. 

Differential  Delay  - A differential  delay  expressed  in  nanoseconds/MHz. 

This  delay  difference  is  from  the  nominal  70  MHz  carrier  for  which  the  propa- 
gation delay  was  determined. 

c.  Measurement  Technique  - A link  with  the  capability  to  handle  13  Mb/s  data 
is  assumed.  At  this  data  rate,  the  magnitude  of  the  variations  is  expected  to 
cover  a significant  number  of  bit  periods  (several  hundred)  and  the  rate  of 
variation  is  expected  to  be  slow  relative  to  a bit  period.  A cesium  standard 
(HP  5061A)  clocks  a PN  generator  providing  a PN  sequence  of  sufficient  length 


13-7 


to  prevent  ambiguities  over  the  expected  delay  variation.  A 70  MHz  sub- 
carrier is  PSK  modulated  with  this  sequence  and  transmitted  across  the 
tropolink. 

At  the  receive  end,  a conventional  PSK  demodulator  and  bit  sync  extract  the 
transmitted  PN  sequence.  This  sequence  is  compared  to  an  identical  sequence 
generated  in  the  same  manner  os  the  transmitted  sequence.  The  PN  generator 
is  clocked  by  a second  cesium  standard.  A simple  range  extractor  circuit  is 
used  to  determine  the  time  delay  between  the  sequences.  A strip  recorder  is 
shown  since  it  will  probably  be  desirable  to  record  variations  in  propagation 
delay  over  long  periods  with  a strip  recorder. 

Note  that  if  both  cesium  standards  are  at  the  worst  limits  of  their  accuracy, 
the  resulting  time  delay  error  will  be  only  72  ns/hr. 

Since  delay  variation  as  a function  of  frequency  will  be  averaged  over  the 
bandwidth  of  the  transmitted  signal,  two  types  of  measurement  are  possible. 

By  using  a relatively  high  bit  rate  the  average  delay  for  the  whole  band  of 
frequencies  may  be  measured.  By  using  a low  bit  rate  (say  10  kb/s)  spot 
checks  of  delay  may  be  made  at  various  points  across  the  frequency  band. 

Figure  13.2-2  shows  how  a swept  (or  stepped)  subcarrier  may  be  added  to 
provide  a means  for  measuring  and  recording  differential  time  delay.  The 
test  setup  used  for  propagation  delay  (Figure  13.2-1)  remains  intact.  In 
addition,  the  cesium  standard  clocks  a frequency  synthesizer  HP  8660C  which 
steps  the  center  frequency  across  the  band  (step  size  and  sweep  rate  are 
selectable). 

At  the  receive  end,  an  identical  sweep  rate  and  step  size  are  generated  with 
a 140  MHz  center  frequency.  A mixer  and  bandpass  filter  are  used  to  extract 
the  difference  frequency  which  is  a constant  70  MHz.  A digital  delay  adjust- 
ment and  a counter  are  used  to  synchronize  the  sweeps  manually.  Once 
adjusted,  drift  will  be  insignificant  due  to  the  high  stability  of  the  cesium 
standard.  A time  marker  from  the  cesium  standard  is  provided  to  the  recorder 
to  provide  a reference  for  frequency  calibration. 


d.  Test  Stimuli  - For  the  propagation  delay  test  the  cesium  beam  time  standard 
driving  the  PN  generator  is  the  prime  stimulus  for  the  test.  For  the  differen- 
tial delay  the  test  stimuli  are  the  PN  sequence  and  a stepped  range  of  fre- 
quencies used  to  offset  the  70  MHz  subcarrier. 

e.  Control  Parameters  - These  parameters  include  the  carrier  frequency  and  trans- 
mitted power  level.  It  is  expected  that  the  received  signal  level  will  be 
maintained  sufficiently  high  to  achieve  a low  BER  during  performance  of  the 
test.  This  test  may  be  repeated  for  different  carrier  frequencies  if  believed 
beneficial  to  support  the  overall  program  objectives,  i,  e.,  network  and  link 
analysis. 

f.  Test  Setup  - Figure  13.2-1  gives  the  test  setup  for  the  propagation  delay  and 
Figure  13.2-2  gives  the  setup  for  the  differential  delay  test.  All  of  the  equip- 
ment required  for  these  tests  should  be  readily  available. 

g.  Data  Gathering  - Data  gathering  on  these  tests  should  consist  of  a plotting  of 
time  delay  on  a strip  chart  recorder.  The  recorder  should  have  an  event 
marker  capability  to  mark  time  hacks.  Identification  of  the  test  and  any  other 
desired  information  may  be  annotated  directly  on  the  recorder  paper. 

h.  Performance  of  Test  - For  the  sake  of  brevity  the  test  will  be  broken  down 
into  a few  basic  steps. 

Step  1 - Gathering  of  Test  Equipment.  -This  consists  simply  of  getting  all 
the  needed  test  equipment  to  the  appropriate  test  sites  at  the  appropriate 
time.  This  includes  the  assembly  of  the  special  test  circuits  (range  extrac- 
tor, recorder  interface). 

Step  2 - Pretest  Checkout.  - A brief  check  should  be  performed  to  verify 
that  each  unit  is  capable  of  performing  its  test  function.  This  includes 
verifying  that  each  piece  of  equipment  has  a current  calibration. 

Step  3 - Connect  Test  Setup.  - Hook  the  equipment  together  as  required 
to  perform  the  test.  The  function  of  the  test  setup  should  be  verified  to 
the  maximum  extent  possible  prior  to  start  of  test  (radio  transmission). 
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Figure  13.2-2.  Differential  Delay  Test  Setup 


Step  4 - Synchronize  the  Cesium  Time  Standards.  - With  the  standards 
side  by  side  the  two  are  to  be  synchronized  to  precisely  the  same  time 
setting.  The  standards  should  then  be  taken  to  the  test  sites  and  connected 
into  the  test  setups.  This  step  may  have  to  be  repeated  depending  on  the 
total  time  span  of  the  testing  effort  in  order  to  preserve  the  close  agree- 
ment of  the  standards. 

Step  5 - Performance  of  Test.  - Using  the  appropriate  carrier  frequency(s) 
and  levels  the  test  is  run  for  a TBD  period.  The  strip  chart  recorder  should 
be  operated  for  the  duration  of  the  test  to  obtain  a continuous  time 
correlated  record  of  the  delays. 

Figure  13.2-3  shows  the  test  setup  for  measuring  the  reciprocal  delay  over  the 
troposcatter  radio  link.  A comparison  to  Figure  13.2-1  shows  that  this  setup  is  basically  a 
full  duplex  version  of  the  propagation  delay  test.  The  prime  concern  here  is  to  obtain  a simul- 
taneous measurement  of  the  propagation  delay  in  both  directions  so  that  the  delay  difference 
for  the  reciprocal  paths  may  be  computed. 

Virtually  all  of  the  previously  discussed  parts  of  the  subplan  for  propagation  delay 
are  applicable  to  the  reciprocal  delay  test.  The  major  difference,  of  course,  is  the  full  duplex 
radio  link  and  the  additional  equipment  needed  for  performing  two  simultaneous  propagation 
delay  measurements.  The  two  radio  transmitters  must  be  operating  with  sufficient  difference  in 
their  frequencies  to  obtain  the  necessary  isolation  of  the  local  transmitter  output  at  the  receiver 
input  via  the  duplexer.  This  difference  in  frequency  will,  of  course,  contribute  a slightly 
different  propagation  path  and  corresponding  delay.  The  remaining  portion  of  the  reciprocal 
delay  differential  is  contributed  by  the  difference  in  delays  through  the  corresponding  equipment 
in  each  path. 

An  alternative  scheme  for  making  a reciprocal  time  delay  measurement  exists  which 
uses  less  equipment.  For  example,  only  one  full  duplex  PSK  modem  would  be  required  instead 
of  two.  This  alternative  technique  would  use  a transponder  approach  and  would  require  separate 
measurement  of  equipment  and  total  propagation  delays.  It  is  expected  that  this  approach  would 
yield  a lower  accuracy  measurement  and  should  be  used  only  if  a sufficient  amount  of  equipment 
is  not  available  to  support  the  primary  test  configuration. 
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ADDITIONAL  EFFORT 


The  recommended  network  synchronization  subsystem  employs  time  reference 
distribution  at  the  highest  levels  in  the  network  and  master-slave  at  the  other  levels.  An 
adaptive  timing  distribution  network  organization/reorganization  technique  will  be  used  at 
all  but  the  tail  ends  of  the  network.  Extremely  narrowband  loops  should  be  used  throughout  to 
maintain  highly  stable  nodal  clocks.  The  techniques  being  recommended  are  not  entirely  new. 

A considerable  amount  of  work  has  been  done  on  several  aspects  of  these  techniques.  This 
section  outlines  additional  work  that  should  be  done  prior  to  any  hardware  development. 

The  application  of  time  reference  distribution  on  the  digital  DCS  has  been  studied 
for  several  years.  One  of  the  more  thoroughly  studied  aspects  of  this  approach  is  the  imple- 
mentation of  an  adaptive  timing  distribution  network  organization/reorganization  technique. 

213 

The  first  technique  for  accomplishing  this  was  proposed  by  Davies  and  Prim.  Modification  of 

170 

this  technique  have  been  proposed  by  Stover  and  Clarkson  College  Investigations. 

92 

In  addition,  the  Canadian  Dataroute  network  uses  a similar  technique.  Since  this  problem 

has  been  so  thoroughly  investigated,  additional  work  on  the  problem  does  not  appear  warranted 
at  this  time.  Master-slave  nodal  synchronizers  are  well  understood  since  several  commercial 
digital  networks  are  using  this  approach.  Therefore,  significant  additional  work  it  not  required. 
Finally,  trade-offs  leading  to  selection  of  loop  parameters  for  both  TRD  and  master-slave  were 
presented  in  Section  3.0,  and  no  significant  refinements  of  these  parameters  should  be  required. 

Flowever,  there  are  several  aspects  of  the  implementation  of  the  recommended 
timing  subsystem  which  should  be  addressed  prior  to  any  hardware  development.  The  imple- 
mentation areas,  some  of  which  necessitate  the  exercising  of  the  computer  simulation  tool 
developed  during  this  effort,  include: 

• Transmission  of  periodic  timing  marks  and  measurement  of  timing  error 

• Technique  for  correction  of  atomic  clocks 

• Technique  for  measurement  of  fixed  asymmetries  and  correction  of  timing 
error  measurements 

• Rate  of  clock  error  measurements  and  criteria  for  clock  correction 

• Time  reference  distribution,  master-slave  interface 


r; 
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The  periodic  timing  marks  transmitted  between  all  pairs  of  TRD  nodes  are  used  to 
estimate  the  phase  error  between  the  local  clock  and  the  network  master.  The  questions  that 
need  to  be  answered  relating  to  the  transmission  of  these  marks  and  the  measurement  of  timing 
error  are  the  following. 

1 . How  should  these  timing  marks  be  transmitted  within  the  expected  multiplex 
structure  of  the  digital  DCS?  Two  possibilities  are  to  let  frame  bits  in  either 
the  Tl  data  stream  or  the  highest  available  multiplex  level  be  the  timing 
marks.  The  advantage  of  using  the  Tl  data  stream  framing  bits  would  be  that 
a single,  common  approach  would  be  used  at  every  node,  while  the  advantage 
of  using  the  framing  bits  from  the  highest  available  multiplex  level  is  that 
better  resolution  can  be  obtained. 

2.  How  should  one  measure  the  error  between  the  received  timing  mark  and  the 
local  clock  phase?  Several  approaches  of  varying  degrees  of  accuracy  and 
complexity  should  be  evaluated. 

3.  Each  combination  of  approaches  for  transmission  of  the  timing  marks  and 
measurement  of  the  timing  error  should  be  compared  on  the  bais  of  complexity 
at  the  desired  level  of  performance.  Each  measurement  approach  will  produce 
a measurement  error  per  sample  that  is  most  likely  greater  than  the  measurement 
error  that  one  can  allow  to  enter  the  loop  filter  without  degrading  the  stability 
of  the  nodal  clock.  The  measurement  error  for  each  of  the  measurement 
techniques  can  be  reduced  to  the  desired  value  by  averaging  a large  number 

of  these  measurements  before  allowing  the  sample  average  to  enter  the  loop 
filter . The  effect  of  the  different  requirements  on  sampling  interval  and 
averaging  time  for  each  technique  should  be  considered  in  the  comparisons. 

The  output  of  the  loop  filter  supplies  a voltage  which  is  to  be  used  in  controlling 
the  frequency  of  the  local  clock.  This  voltage  can  be  used  directly  to  control  the  frequency  of 
quartz  crystal  VCO's.  However,  voltage-controlled  frequency  adjustment  of  available  cesium 
and  rubidium  atomic  frequency  standards  is  not  as  easy.  Several  alternatives  for  providing  this 
frequency  control  should  be  investigated. 
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1 . There  is  a rubidium  frequency  standard  available  from  Efratom  which  allows 
voltage  control  of  the  output  frequency.  In  addition,  one  could  make  modifi- 
cations to  HP  cesium  standards  to  allow  vohage  control  of  its  frequency.  Thus, 
one  option  is  the  direct  control  of  the  frequency  of  the  atomic  standard. 

2.  Another  possible  solution  is  the  use  of  an  outboard  phase  shifter.  The  function 
of  this  unit  would  be  to  slightly  shorten  or  lengthen  the  clock  pulses  coming 
from  the  frequency  standard  to  produce  a small  change  in  frequency.  The 
amount  of  this  change  should  be  directly  controlled  by  the  voltage  from  the 
loop  filter.  The  advantage  of  this  approach  would  be  to  avoid  any  modifica- 
tions of  frequency  standards  to  allow  voltage  control  of  frequency.  However, 
to  achieve  the  very  small  changes  in  frequency  that  are  desired  may  be 
difficult  to  do. 

The  performance  of  TRD  may  be  improved  by  measurement  of  asymmetries  in  equip- 
ment delays  and  utilization  of  this  information  in  correcting  the  estimated  phase  error  between 
the  local  clock  and  its  reference.  It  would  be  very  beneficial  to  devote  some  time  to  determining 
typical  values  for  these  asymmetries,  the  best  techniques  for  measuring  them,  and  an  approach  for 
correcting  the  estimated  phase  error  between  the  local  clock  and  its  reference.  Fixed  asymmetries 
will  most  likely  be  very  small,  and  measurement  of  them  will  be  needed  infrequently.  However, 
the  removal  of  these  asymmetries  from  the  phase  error  estimate  can  result  in  better  transient  per- 
formance during  acquisition  of  a new  reference. 

The  rate  at  which  clock  error  measurements  should  be  made  and  the  criteria  for 
making  a clock  correction  is  a function  of  the  types  of  clocks  in  the  network,  the  network  con- 
nectivity and  the  transmission  bit  rates.  The  computer  simulation  that  has  been  developed  will 
be  a valuable  tool  in  selecting  an  optimum  approach  as  well  as  evaluating  various  options  for 
processing  error  measurements . 

The  selected  subsystem  employs  both  the  time  reference  distribution  and  master- 
slave  techniques.  Where  one  technique  leaves  off  and  the  other  begins  is  still  a loosely  defined 
point.  Obviously  this  is  heavily  dependent  on  network  connectivity  patterns.  Again  the  computer 
simulation  tool  should  be  used  to  great  advantage  to  evaluate  the  performance  at  this  interface  and 
determine  criteria  for  its  placement. 
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If  is  believed  that  the  areas  discussed  above  should  be  included  in  future  work  on 
time  reference  distribution.  Once  this  work  is  done,  there  are  also  several  minor  problem  areas 
that  should  be  considered: 

• Refinement  of  design  for  the  TRD  phase-locked  loop. 

• Define  interface  between  master-slave  and  TRD  nodes  and  the  method  for 
measurement  and  report  of  buffer  asymmetries  of  master-slave  nodes  to  TRD 
nodes . 

• Quantification  of  data  transfer  and  processing  requirements. 

Investigation  of  each  of  these  problem  areas  should  allow  the  network  synchronization  to  be 
very  well-defined. 
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FREQUENCY  ACQUISITION 

Introduction 

One  important  aspect  of  a network  timing  subsystem  is  its  response 
to  transient  conditions.  The  purpose  of  the  present  exercise  is  to  investigate 
the  response  of  individual  nodes  to  unit  step  transients  under  noiseless  conditions. 
The  major  parameters  to  be  observed  are  the  size  of  buffers  needed  to  avoid 
loss  of  data  and  the  amount  of  time  needed  to  overcome  the  transient.  A compar- 
ison of  the  relative  acquisition  performances  of  a one  cycle  non  linear  sinusoidal 
phase  detector  versus  an  extended  range  linear  phase  detector  such  as  might  be 
obtained  by  using  buffer  fill  will  be  made. 

A2  Non  Linear  Noiseless  Sinusoidal  Model 

A2.1  Block  Diagram 

A block  diagram  of  a phase  lock  loop  utilizing  a non  linear  sinusoidal 
phase  detector  is  shown  in  Fig  A1 . From  this  figure,  it  should  be  observed  that 

•I 

the  error  voltage  applied  to  the  VCO  is  a sine  function  of  the  difference  in  phase 
between  the  reference  signal  and  the  VCO.  For  this  reason  the  phase  comparator 
is  called  a non  linear  sinusoidal  phase  detector.  Due  to  this  non  linearity,  exact 
analyses  are  not  available  and  approximations  must  be  utilized. 

A2.2  The  Relevant  Equations 

The  following  derivations  of  lock  range  and  capture  range  are  from 
Moschytz  C 54  ] . 
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I tick  Range  - The  differential  equation  describing  the  dynamics  of 
the  loop  of  Fig  A1  is 

d 4>  (t)  d 4>(t) 

j—  = — — *•  K h(t)  * sin<j>(t)  (Al) 

d t d t 

where  4>(t)  = 4>.(t)  - 4>^ ( t ) , K is  the  open  loop  gain,  h(t)  is  the  impulse  response 
of  the  low  pass  filter  and  indicates  convolution.  For  unity  dc  gain  of  the  low 
pass  filter  and  A uk  - d<J>./dt  equation  (Al)  has  steady  state  solution 

sin4>  - A uj^/K  (A2) 

Thus  it  is  possible  for  the  system  to  obtain  a steady  state  condition  with  zero 
average  frequency  error  between  the  reference  signal  and  the  local  VCO 
provided 


I A uui  | « Kt  (A3) 

A2.3  Capture  Range  - The  capture  range  is  that  range  of  input  frequencies 

to  which  the  VCO  can  be  synchronized  to  when  initially  in  the  unlocked  state.  If 
the  loop  filter  were  perfect,  i.e.  , passed  the  low  frequency  difference  components 
without  attenuation  and  rejected  the  high  frequency  components  from  the  phase 
comparator,  then  the  capture  range  would  exactly  equal  the  lock  range.  Since 
practical  filters  are  not  perfect,  the  capture  range  is  reduced  from  that  of  the 
lock  range.  Determining  the  actual  capture  range  involves  the  solution  of  a non 
linear  differential  equation.  The  only  known  method  for  doing  this  is  by  means  of 
a graphical  technique  involving  a so  called  "phase  plane  method.  " However,  an 
estimate  of  the  capture  range  can  be  obtained  by  determining  for  an  input  offset 
frequency  A uu  the  amount  of  steady  state  error  voltage  at  the  VCO  input 
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terminals  under  closed  loop  conditions  and  comparing  this  with  the  peak  ac  error 
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Capture  Range  for  Specific  Filters 


L 


For  the  low  pass  filter  of  Fig  A2-a  the  attenuation  as  a function  of 
radian  frequency  is  given  by 


H - 1/(1  + w3  T< 

W 


(A9) 


Substituting  (A9)  into  (A8)  we  obtain 


A wic  * ^VC( 


(A10) 


1 


Where  m is  the  3 dB  cutoff  frequency  of  the  low  pass  filter  and  equals  — 
co 


I , T x 

The  term  inside  the  braces  is  of  the  form  V a3  + 1 - a.  But  0sya3+l-ail 
for  all  real  positive  a.  Thus 


A Wic  * ^Vco 


(All) 


For  the  low  pass  filter  of  Fig  A2-b  the  attenuation  as  a function  of 
radian  frequency  is  given  by 


H = 

(li 


l + w2^3  1 £ 


l + (Da  T* 


(A12) 


Substituting  Aw-  for  W in(A12)and  substituting  the  resulting  factor  into  (A8), 
solving  for  Aui  and  then  substituting  2 £ wn  = (1+K^,t  t )/t  8 and  uo^3  = K^,/  t a 


in  the  solution  we  obtain 
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— — - — 


(A13) 


For  — - < < Z the  term  inside  the  braces  may  be  approximated  by 
K 

T 

Jz  E)a  + V(2  f.3  )3  + l 


By  choosing  the  time  constants  t x and  t s of  Fig  A2-b  in  a 
suitable  manner,  this  circuit  can  be  made  to  approximate  that  of  the  integral 
plus  proportional  filter,  Kp+Kj/s.  This  is  true  provided  >>  — where 

K^,  is  the  open  loop  DC  gain. 


For  the  integral  plus  proportional  filter,  the  frequency  response 


function  is 


H = 
U) 


l + U)3  T,  a1  fc 


Substituting  this  factor  into(A8)we  find  that  the  capture  range  is  given 
approximately  by 


Auu.  uu  V 2 P a + •/  (2  Pa  )*  + 1 
ic  n 


(A  14) 


A2.4 


Closed  Loop  Noise  Bandwidth 


Two  sided  closed  loop  noise  bandwidth  is  defined  by 


WL=  2 


hi; 


F(s)  1 a d s 


(A  15) 


One  sided  noise  bandwidth  = \ 
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For  the  basic  phase  lock  loop,  the  closed  loop  phase  transfer  ratio 


is  given  by 


F ( 8 ) 


KtH(8) 
a + K H ( s ) 


(A  16) 


where  H(s)  is  the  low  p.'iss  filter  transfer  function  and  K is  the  open  loop  DC 
gain.  The  integral  in  (A15)  may  be  evaluated  by  the  method  of  complex  residues 
or  by  partial  fraction  expansion  if  F(s)  is  a rational  function.  A short  table  of 
integral  forms  in  the  rational  case  is  given  in  [56],  p.  135.  Evaluation  of  (A 75) 
for  four  types  of  filters  gives  the  following  results: 


H(s)  = 1 -♦  Wj  = K /2 


H(s)  = 
H(s)  = 

H(s ) = 


1 

II 

£ 

K uu 

T n 

1 + T x a 

2 4 F 

1 + Tx  s 

WL  = 

Kt(ts+Ti2Kt)  / 

4 5 2 + 1 

1 + T 3 s 

-4 

2ts(Kttx  + 1)  " %\ 

4 ? 

1 + T x 8 

W L = 

+ KtTi  - . ( 

4 f;3  + 1 

s 

-♦ 

2 t x 2 n \ 

4 ? 

for  tx  >> 


K r 


^2  5 Time  to  Acquire  Synchronization  - For  a second  order  loop  with  an 

integral  plus  proportional  loop  filter  of  the  form 


H(s ) = 


T 


A 


f 


(~-) 

TT  2 / 1 + 4 F3  \ 3 
4 F \ 4 ^ / 


lAiil 
B 3 
L 


2 it  3 

? 


iAlll 


(A  17) 
(A18) 


where  &f  is  the  initial  offset  frequency  in  Hertz 
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• ■*** 


I 


V 


4 


bl  = 


uu  (4  +1) 

n 


4 T 


= one  sided  loop  noise  bandwidth 


,1  + 4 ? 8 ) = 8 

^ is  the  loop  damping  ratio, 

2 P 

T , = is  the  time  constant  of  the  loop  filter,  and 

^n 

ID  is  the  undamped  natural  frequency  of  the  second  order  loop, 
n 

Derivations  of  equation  (A18)  can  be  found  in  Richman  [55] 

Lindsey  [56]  chapters  3,  4,  and  10.  Stiffler  [57]  chapter  5 derives  (A18)  but 

states  that  there  are  no  theoretical  limit  If  on  the  capture  range.  It  should  be 

observed  from  (A  18)  that  the  time  needed  to  acquire  frequency  synchronization 

grows  as  the  square  of  the  initial  frequency  offset  £f.  This  fact  will  be  used  to 

obtain  an  expression  for  the  frequency  of  the  error  signal  as  a function  of  time 

during  the  pull  in  process.  Such  an  equation  allows  calculation  of  the  buffer  size 

required  to  hold  all  the  bits  slipped  following  a step  transient  in  input  frequency 

to  the  PLL.  After  frequency  synchronization  has  been  obtained,  some  additional 

time  T is  required  to  obtain  phase  lock.  T can  normally  be  approximated  by 
4>  9 


T ~ 4/B 

<J>  JL 


(A19) 


» 


Since  T is  small  in  comparison  to  that  needed  for  frequency  synchronization 
9 

when  a f is  very  large  and  no  further  cycles  are  slipped,  this  term  will  normally 
be  neglected. 

Example  of  time  to  acquire  synchronization  using  (A  18) 


« 


Let  <t  = -y~r  , and  Af  = 10~8f  =.  01544  Hz 

y 2 O 


From  (A14)  the  capture  range  is  sr  1.55 


=>wn  = 


. 01544 
1.  55 


X 2ir 
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Therefore  let  ua  assume  that 


- .0628  rad/sec.  Then 


n 


bl  = 


uun(4  ?a  + 1) 
8 F 


and 


a \ 3 


TAf 


TT  3 / 1+  4 F 3 \ 

' 4 ? \ 45  ) 


0628  (3) 
5.  656 


lAlil 

V 


^ . 033  Hz 


4.  16  x 


2.  384x  10 

3.  696  x 10 


= 26. 83  sec. 


Actually,  in  order  to  filter  out  link  disturbances,  a value  of  w on  the  order  of 

-5  n 

5.6  x 10  rad/sec  would  be  required.  Thus,  reacquiaition  of  synch ronizati on 

following  the  above  step  change  in  reference  frequency  would  first  require  a 

- 5 

change  in  bandwidth  of  at  least  .0628/5.6  x10  = 1121  times  that  of  the  normal 

tracking  bandwith.  Step  changes  in  reference  oscillator  frequency  of  the  size 
considered  in  this  example  can  be  expected  in  a timing  subsystem  when  switching 
from  one  reference  oscillator  to  another  or  as  a result  of  abnormal  link  disturbanc 


A2.6  Frequency  Acquisition  Techniques 

From  the  above  example,  it  should  be  apparent  that  due  to  the  size  of 
transients  and  the  required  loop  time  constants  for  filtering  out  normal  link 
disturbances,  some  scheme  is  required  for  obtaining  initial  synchronization  and 
for  handling  abnormal  transients.  Three  major  approaches  have  been  suggested 
for  accomplishing  this  objective.  They  are  as  follows: 

1.  Provide  two  filter  bandwidths,  a narrow  one  for  tracking  and  a 
much  wider  bandwidth  for  initial  acquisition. 

2.  Use  a sweeping  or  search  technique  during  acquisition  to  force 
the  local  VCO  to  approximately  the  correct  value  prior  to  final 
acquisition. 


1 


3.  Use  an  extended  range  linear  phase  detector  such  as  might  be 
provided  by  buffer  fill  in  order  to  develop  a large  error  signal 
that  is  proportional  to  the  number  of  bits  slipped  between  the 
local  VCO  and  the  input  reference. 

It  may  actually  turn  out  that  some  combination  of  the  above  approaches  is  the 
best  solution  for  a particular  timing  subsystem.  The  remainder  of  this  section 
will  investigate  the  first  two  approaches  while  the  next  section  will  consider  the 
third  option. 


A2.6.1  The  Sweep  Technique  for  Acquiring  Synchronization 

This  technique  consists  of  applying  a sweep  voltage  directly  to  the 
VCO  input  and  removing  it  as  soon  as  frequency  synchronization  is  detected. 

' It  may  be  shown  that  if  the  sweep  rate  is  greater  than  the  total  DC  gain  of  the 
loop  (K^,),  the  system  cannot  lock  onto  the  reference  signal,  but  if  the  rate  is 
leas  than  K^/2,  then  lock  can  be  obtained.  At  lower  signal  to  noise  ratios,  the 
sweep  rate  must  be  smaller  than  for  the  ideal  rate,  but  for  many  practical 
problems  can  be  almost  K^,/2.  For  the  sake  of  this  discussion,  let  us  assume 
that  a sweep  rate  of  K^,/4it  is  suitable.  Then  frequency  acquisition  time  is 
given  by 


T>  = 4*  Af, 

kt 


rr  kI 


( 1 + 4 \ s) 


sec. 


(A20) 


V 


1 - 4 

with  5=  , £f  = . 01544  Hz  and  BL  = 2.0833  x 10  Hz  corresponding  to 

a time  constant  t t of  1 hour 


/ 


= 1.2573  x 10  sec  = 14.55  days. 


Thus,  it  appears  that  the  sweep  technique  alone  is  unsuitable  for  acquiring 
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r 


I 

synch  ronization  with  bundwidths  corresponding  to  time  constants  on  the  order  of 

- 8 

an  hour  if  the  frequency  offset  is  on  the  order  of  10  x f . Comparison  of 
equation  (A20)  with  (A18)  indicates  that  the  sweep  technique  can  become  effective  if 


A2.6.2  Two  Bandwith  Approach 


The  amount  that  the  loop  bandwidth  can  be  increased  i6  generally 
limited  by  noise  considerations.  However,  due  to  the  extremely  narrow  tracking 
bandwidths  being  considered  and  the  high  signal  to  noise  ratios  that  can  be 
expected  f o r a communication  network  timing  subsystem,  this  may  not  be  a real 
problem.  Neve  rtheless,  the  acquisition  bandwidth  must  be  great  enough  that  the 
maximum  input  frequency  deviation  from  that  of  the  local  VCO  is  within  the  PL.L  s 
capture  range  if  this  method  is  to  be  successful.  From(AlO),  (A13),  or  (A  14). 
depending  on  the  loop  filter  chosen,  we  may  choose  values  for  closed  loop 
parameters  uu  and  f which  will  insure  that  the  PPL  will  acquire  synchronization 
for  a given  reference  frequency  offset.  Fig  A3  shows  1 and  bits  slipped 

during  frequency  acquisition  versus  F when  the  loop  parameters  are  adjusted 
for  a capture  value  of  £f  - 5 x l(fH  x f = .0772  Hz  for  the  integral  plus  propor- 
tional filter.  From  this  figure,  it  may  be  observed  that  the  bandwith  required 


* 

* 

*i 

•• 

* 

% 

«* 

r 

i . 


by  capture  equations  (A8)  and  (A14)  is  probably  larger  than  actually  required  for 
acquisition  since  under  this  condition  no  bits  will  be  slipped  for  F <1.5. 

An  implementation  problem  for  an  automatic  two-bandwidth  approach 
is  to  devise  a mechanism  which  determines  when  to  switch  from  tracking  loop 
parameters  to  acquisition  parameters  and  vice-versa.  Basically,  this  reduces 
to  determining  the  state  of  the  VCO  error  voltage,  i.e.  , whether  it  is  at  steady 
state  d<  corresponding  to  a locked  state  or  has  an  ac  component  corresponding 
to  an  acquisition  state.  This  will  be  pursued  further  for  the  extended  range 
linear  detector. 
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F rom  Fig.  A3  it  is  observed  that  for  ^ = 4,  uj  & . 057  rad/s  if 

n 

the  loop  is  to  capture  from  an  input  frequency  offset  of  . 0772  Hz.  In  order  to 

adequately  filter  out  daily  variations  in  link  propagation  delays  in  a long  chain 

-5 

of  timing  nodes,  a value  for  ^ on  the  order  of  5.  6 x 10  rad/s  and  P = 4 

n ^ 

is  required.  Thus,  in  order  to  be  within  the  capture  range  with  an  offset  of 

-8  3 

5 x 10  x fQ,  must  be  increased  by  a factor  of  approximately  10  if  the  same 

damping  ratio  is  maintained. 


Number  of  Bits  for  Non-Linear  Model  Pull  in  Strategies 


A2.7.1  Natural  Pull  in 

If  we  assume  that  the  frequency  difference  remained  at  the  initial 
difference  during  the  entire  transient,  i.e.,  number  of  bits  = NB  = Af.  T 

Af 

Then  our  estimate  would  be  too  large  because  the  local  VC O frequency  begins 

to  be  pulled  toward  the  input  reference  signal  as  soon  as  the  transient  begins. 

The  rate  at  which  the  frequency  changes  accelerates  during  the  pull  in  period 

as  is  evidenced  by  the  fact  that  T is  proportional  to  (A  f)2  . Conversely, 

A f 

if  we  assume  that  NB  - 1/2  Af  • T , then  our  estimate  would  be  too  small. 

Af 

F'or  example,  if  it  takes  T seconds  to  achieve  frequency  synchronization 

when  the  initial  frequency  offset  is  fcf  and  pull  in  time  is  proportional  to 

(Af)2  then  the  time  required  to  pull  in  from  1/2  Af  is  1/4  T . Thus  the 

A f 

time  required  to  go  from  Af  to  1/2  Af  Is  3/4  T . We  may  then  write 

A f 


t - t - n ( a f(t ))a 

A 1 


where  f!  is  a constant  of  proportionality.  Then 


(A21 ) 


A- 15 


making  a change  of  variable  y = T - t,  - dy  = dt 

A1 


r 


(A22) 


(A23) 


A2.7.2  Sweep  Assisted 

If  a sweep  voltage  is  applied  that  results  in  a frequency  change 
rate  that  is  as  high  as  the  PLL  can  lock  to  as  the  phase  detector  output  signal 
frequency  passes  through  zero,  then  we  may  assume  that  the  average  frequency 
of  the  error  signal  over  the  acquisition  period  is  £ Af-  In  this  case 

NB  = i Af  • T (A24) 

* At 

A3  Linear  Model  With  Extended  Range  Phase  Detector 

A3.1  Development  of  the  Model  and  Its  Solution 

A linear  model  for  the  PLL  may  be  obtained  by  using  an  extended 
range  phase  detector  such  as  buffer  fill  or  an  up-down  counter  provided  the 
buffer  or  counter  is  chosen  large  enough  to  hold  the  largest  transient.  This 
will  be  assumed  for  the  following  analysis.  Under  this  assumption,  the  objective 

A-16 
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is  to  determine  the  time  required  to  obtain  synchronization  and  the  amount  of 
buffering  required  when  a etep  difference  in  frequency  Af  is  applied  at  the 
input. 


A3. 1.1  Phase  Error  Equation 

Using  the  second  order  loop  shown  in  FigA4  with  a proportional 
plus  integral  filter  we  may  obtain 


Bf(9)  1 _ 1 

F.  (s)  " s +K  K H(s)  " s+K  Ut,s 

1 Ij  V 


8a  +KtT  t b +Kt 


(A25) 


Thus , 


where  K = K K and  t , is  the  time  constant  of  the  filter. 
t B V 1 


8 F.(s) 

Bf(8)  = sa  +2  Puu  S +-J0  3 


(A26) 


where  uj  - V K and  ? = (t  ^ '/k  )/Z 

n T 


Now  letting  F.(s)  correspond  to  a step  function  of  magnitude  Af  starting  at 
time  t-0  we  obtain 


Af 

Bf^8*  S 3 + 2 P ijj  8+yt  3 

- n n 


(A27) 


The  poles  of  this  function  are 

B - - 5 UU  * (!)  7 «! 8 - 1 

n n 


A- 17 


The  following  is  noted 

(1)  P > 1,  both  poleB  are  negative  and  real. 

(2)  5=1,  the  poles  are  equal,  negative,  and  real  (b  = - yj  ). 

(3)  0 < 5 < 1,  the  poles  are  complex  conjugates  with  negative  real 

parts  (s  = - 5 uj  ± j aj  V 1 - § a ). 
n n 

(4)  P = 0,  the  poles  are  imaginary  and  complex  conjugates 

(5)  5 < 0,  the  poles  are  in  the  right  half  of  the  s plane. 

Cases  (4)  and  (5)  are  oscillatory  or  unstable  and  will  not  be  considered  further. 
For  case  (1)  the  solution  is 


A f -at  -b  t 

bf(t)  = b^T  (e  -e  > 


(A28) 


Where  a 


w. 


( 5 + V ?a  - L ) , b = u)  ( 5 - V ? a - 1 ) 


For  case  (2)  the  solution  is 


bf(t)  = te  ,J,nl 


(A29) 


For  case  (3)  the  solution  is 


b,(t)  = e - sin.jj  t 

f uu,,  d 


(A30) 


where  ui  . - uu 
d n 


sr 


A3. 1.2  Output  Frequency  Equation 

F rom  Fig  A4 
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Fo<»>  KBKVH(a) 

Fi(s)  ' s + K0K v H( a ) 

For  the  integral  plus  proportional  filter 

Fo(a)  kt(1  + t1b)  Kt(1  + tx«) 

F.(s)  ” s3+Ktt1s+Kt  aa  + 2 ? + u)^3 

where  K^,,  Tj,  P.  and  are  the  same  as  in  (A25)  and  (A26). 

F.(s)  be  a step  of  magnitude  Af  at  time  t=0  we  obtain 

AfKT(l  + Tl  s) 

V ^ 8 ( 8 3 + 2 P W s+uu  a) 

n n 


A f uu  3 

n 


2 a f uu  ? 


n n 

s(aJ+2f  u s + id  a)  + s3  + 2 P uu  8 + u)  3 
J n n n n 


The  corresponding  time  domain  equations  are  as  follows: 


( 1 , /—a — Tv  - (P  +7  - 1)  uu  t 

5 > i.  f0(t)  = Af  h -zjrrri  +V?5,7T) e ' v 

/ - /m  v - ( p - - 1 ) uu  t 

- (s  - J7r  - 1 ) e n 


? = 1,  f (t)  = Af  1 - e “n1  + 11)  t e 
3 o n 


0 < % < 1 , fQ(t)  = Af  jl  + Jl  J '^5  e"  " 2 P sinj^t  - sin^t^-^)  j 


where  uu.  = U)  Vi  - P3  and  <}>  = cos  5 

d n 
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. A3.2  Time  of  Peak  Phase  Error 

i 

In  order  to  determine  the  time  t ^ corresponding  to  the  maximum 

value  of  bf(t)  we  take  the  derivative  of  bf(t)  and  set  equal  to  zero  and  solve 

for  t . ThuB  we  obtain: 

m 


E > 1 , t 

^ m 

1 

a-b 

L n 

(f) 

(A37) 

where  a = u) 

n 

and 

b 

= lDn  ( ? - 1 ) 

5=1,  t 

* m 

1 

ID 

n 

(A38) 

£ 

•p 

V 

V 
o 

1 

Wh 

cos 

'\v 

(A39) 

whe  re  yj  , - o> 
d n 


VT 


A 3 3 Peak  Phase  Error 

Substituting  (A37),  (A38)  and  (A39)  into  (A28),  (A29)  and  (A30) 
respectively,  we  obtain  peak  phase  errors. 


> 1 bf(tm)  = b-a 


Ai_  |/±_|  b'a  . (-*-)  b'a 


(A40! 


where 


a = uu  ( ? + - l ) and  b = Z 'V?*  ~ 1 ) 


e = 1 b (t ) 


A L 


f m e uu 


(A41 ) 


■1 


_Al  -O'  cos  ? ) / V1  - 

0 < E < 1 b (t  ) = e 

^ f m id 

n 


(A42) 
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A 3.4 


Settling  Time 


The  time  to  Bettle  following  a step  change  in  input  reference  frequency 
may  be  conveniently  defined  as  the  time  at  which  the  transient  has  died  to  within 
some  fractional  part  p of  its  peak  value.  In  practical  situations  p is  often 
chosen  as  .01.  The  desired  relations  are  as  follows: 

a b , 


? > 1 , 


-at 
e s 


e 


- bt 


8 


£ 


(A43i 


where  a = uj  ( ^ 4 J P ^ - 1 ) and  b - uu  ( ? ' «/  j 3 " ^ ) 

n n 


? = 1 


-UU  t P 1 

uj  t e n s s — , t > 


n s 


8 H) 


- f id  t /r — 5-  - cos  )/yi- P3 

0 < % < 1 , e 2 % s sinuu  t * P -Z1  " e 

Q 8 


?A44) 

(A45) 


For  ^ > 1,  the  equations  are  not  readily  solved  by  algebraic  methods. 
However,  simple  plots  of  the  left  hand  sides  provide  answers  for  particular  cases. 
For  P > 1.  8 the  ratio  of  the  two  poles  is  > 10.  Hence,  in  this  case  the  left  side 
of  (A43)  can  be  approximated  by  the  dominant  pole  term,  e.  g.  the  term  involving 
the  smaller  of  a and  b.  For  0 < p_  < 1,  the  sinusoidal  factor  on  the  left  side 
of  (A45)  can  be  ignored  (approximated  by  its  peak  value),  thereby  approximating 
the  response  by  the  exponentially  damped  envelope. 


i A3. 5 Time  Of  Peak  Output  Frequency 


solving  for 


Differentiating  equations  (A34)  - (A36)  and  setting  equal  to  ze  ro  and 
t we  find  that  the  time  of  peak  frequency,  t is  given  as  follows. 
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e > l.  t 


pf  U) 


1 / e \ \ 

rrrr r L“  ( ) 


? = 1 . t 


pf  ID 


l . -l  / 2 e.Ji 
0 < ? < 1 • 'pf  = “Ji"?  \ 2 ?3 


(A  46) 
(A47) 
(A48) 


A3. 6 Peak  Output  Frequency 

Subsisting  (A46)  - (A4B)  into  <A34)  - (A36)  respectively,  we  find  that 

peak  output  frequency  (fQp)  ia  as  follows:  2a^  2 b (JJ 


\ % , a . b-a  , a_v  b-a 

? > 1 • fop  = Af  I + b-a  a(  b ) ( b ) 


(A49) 


where  a - id 


( C + J 7,  ^ * ) and  b ~ id  n ( " V p. 13  “ 1 ^ 


.si  £ = Af  (1  + e ) 3-  1.135335  Af 

^ ’ op 


(A50) 


o < e <i,  f - 

^ • op 


F g ____  ^ ) 

1 + 1 e ‘ /TTTa  ? sin  008  & | (A51) 


R f 

• t 

X *\ 

whe  re 

, AsJUs! 

g = tan  2 ? 2 - 1 

ft 

I * 

A3. 7 

Exan.ple  of  Settling  Time 

K ** 

1 * 

B?  l' 

Let  5 = 4,  u>n  = 5.  6 x 10 
From  (A37)  t^  - a b L 

I ** 

a = 5.6x10  (4  + «/T5~ ) = 

ft  • 

b = 5.6x10  S(4-  yiT  ) = 

f=5xl0  x f - 
1 o 


= . 0772  Hz , and  p = . 01. 
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b-a  = - 4. 3377  x 10 


a/b  = 61.  98 


-4 

then  t = 9514  sec 
m 

From  (A40)  peak  phase  error  is 

b (f  ) = 163.  6 bits 
f m 

Usiny  (A43)  and  approximating  the  left  hand  side  by  the  dominant  pole, 

t 2 659279  seconds, 
s 

From  this  example,  it  is  evident  that  the  system  will  not  acquire 
synchronization  very  quickly  with  the  tracking  loop  parameters  designed  to 
filter  out  daily  path  delay  variations.  Thus,  an  investigation  of  a two  bandwidth 
scheme  for  the  linear  model  will  be  made  next. 

A3  g Two  Bandwidth  Linear  Model 

A3. 8.1  Motivation 

In  order  to  reduce  buffer  size  using  the  linear  model,  two  band- 
widths  may  be  provided,  one  for  normal  tracking  and  one  for  fast  acquisition. 
In  the  normal  tracking  mode  each  node  should  track  the  long  term  average  of 
the  incoming  frequency,  but  filter  out  normal  link  perturbations  and  clock 
jitters.  It  is  also  desirable  to  provide  the  capability  of  flywheeling  through 
short  link  outages  without  slipping  bits.  The  purpose  of  the  fast  acquisition 
mode  is  to  quickly  acquire  synchronization  following  large  abrupt  changes  in 
input  frequency  such  as  during  initial  system  startup  or  following  failures  in 
which  a new  reference  must  be  obtained.  Once  the  phase  lock  loop  parameters 
have  been  chosen  to  accomplish  the  above  system  dynamics,  the  task  then 
becomes  one  of  determining  criteria  for  switching  from  the  tracking  mode  to 
the  fast  acquisition  mode  and  vice-versa. 
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A3. 8. 2 


A Simple  Mode  Switching  Technique 

A simple  technique  for  accomplishing  the  mode  switching  task  is  to  switch  from 
normal  tracking  to  fast  acquisition  as  soon  as  phase  error  exceeds  some  threshold  amount 
which  is  above  that  which  can  be  expected  for  normal  link  perturbations  and  clock  jitter  and 
drifts,  and  to  switch  from  fast  acquisition  mode  to  normal  tracking  mode  after  having  been  in 
fast  acquisition  mode  for  an  interval  of  time  Tfa  sufficient  to  reduce  frequency  offset  lo  within 
a specified  tolerance.  is  determined  by  the  acquisition  mode  loop  time  constants  and  by 

the  amplitude  of  the  initial  frequency  and  phase  error.  The  following  analysis  is  made  to 
determine  suitable  values  of  / ^ and  T^  . 

Study  line  of  sight  and  cable  link  perturbations  and  clock  drifts  indicate  that  the 
combined  equivalent  normal  maximum  frequency  offsets  under  normal  tracking  conditions  shoula 
be  less  than  5 x lCf 10  x f.  Scaling  from  example  A3. 7,  this  corresponds  to  a peak  buffer  fill  of 
1 .09  ps.  For  large  short  term  variations  such  as  with  tropo,  maximum  propagation  time  variation 
may  be  as  much  as  3.3  ns/mile  over  a 1 minute  interval.  For  such  a short  time  the  loop  will  not 
respond,  so  a phase  error  will  result.  A 300-mile  link  would  then  give  a total  time  variation  of 
0.99  ps , which  would  result  in  0.99  x 1 .544  = 1 .53  bits.  A 400-mile  link  would  result  in 
1 .32  ps . Expected  maximum  tropo  link  distances  are  300  miles.  Normal  satellite  variation 
should  result  in  an  approximately  8.42  ps  daily  variation  clock,  but  it  will  be  assumed  that 
techniques  are  available  for  removing  all  but  about  1 percent  of  this  variation.  Thus,  timing 
network  nodes  connected  to  satellite  links  can  expect  to  see  a daily  variation  of  approximately 
±8.42  ps  variation  1 .544  Mb/s  rate  when  referenced  to  the  input  frequency.  If  it  is  assumed 
that  this  accumulation  of  phase  error  occurs  at  a constant  rate  over  a 12-hour  period,  then  the 
equivalent  frequency  offset  is  1 .95  parts  in  10  . 


This  implies  an  instability  of  s - 3.009  x 10  /I  .544  x 10  - 1 .9488  x 10 

Thus,  the  peak  phase  error  would  be  less  than  .673  bits.  If  the  accumulation  is  over  a 4 hour 
period  rather  than  12,  then  the  equivalent  instability  is  5.85  x 10  which  results  in  a peak 
phase  error  of  1.98  bits. 


A3  8.3  Buffer  Fill  In  Fast  Response  Mode  With  Non-Zero  Initial  Phase  Error 

When  switching  from  the  normal  tracking  mode  to  the  fast  acquisition 
mode,  after  a phase  error  b^  has  been  built  up  following  a step  change  in 
frequency  Af.  The  response  is  not  the  same  as  it  would  be  with  zero  phase  error. 
In  this  case 


Bf(s) 


s(F.(s)  - bQ) 

8s  + 2 t U)  8 + U)„a 

n n 


Af  - sb 


s3  + 2 ? tDn8  + uon 


(A52) 


and 


whe  re 


and 


b (t ) = sinuJ.t 

f utd  d 


,J)d  = “n 


■JT. 


a = ( ? +V  -l)tu 

b = ( e - V ?3  -i)i 


(a-b) 


-at  -bt 
(a  e -be  ) 


(A53) 


For  P = 1/-/T  this  becomes 

r ] ' 

b (t)  = e * ■ ^n1  Bind)  + b L e * ? V (sin  ui  ft  - cos  u,  ?t)  J (A54) 

f (ji  , do  n 
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From  this  equation  it  should  be  observed  that  the  initial  buffer  offset  is 


exponentially  damped  with  time  constant  1 / ? which  is  the  same  as  that 

n 

for  the  initial  frequency  offset. 

A3. 8. 4 Frequency  In  Fast  Response  Mode  With  Non-Zero  Initial  Phase  Error 

In  the  case  of  non  zero  initial  phase  error  upon  switching  to  the 
fast  acquisition  mode,  we  have 


2 Suu  s +ui  “ 

n n 


F (s)  = (F.  (s)  - b ) x -a  A 2 

o 1 o Sa+2Pil ,8+r-  2 


ai  * + U) 
n n 


(A55) 


Letting  f.  (t)  be  a step  of  amplitude  frl  beginning  at  t = 0 


F (s)  = 
o 


(■*  - -.) 


2 ?U)  8 +UJ  * 

n n 


a2  + 2f u)  s + id  2 
’ n n 


(A56) 


For  5 < 1 and  g 
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f(t)  = e ^ ain  dj  t n uu  ” uu  uu  , 
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whe  re 


ID 


. = VT-“ 


In  case  5 = 1 /•fz 


this  reduces  to 


f(t)  = A f 1 1 + e ^UJnt[  sin  t • cos  C ui  t J } 

n n 

■ 25%b0  e ^n*  cos  ^ t (A58) 

- 8 

F rom  (A58)  if  A f is  equal  to  10  x f = . 01544,  P = , 707,  and  uj  = . 014, 

o ’ n 

then  the  transient  due  to  the  initial  frequency  offset  has  an  initial  amplitude 
-8 

of  10  while  that  due  to  the  initial  phase  offset  has  initial  amplitude  of  . 02  b . 

o 

A3. 8. 5 Strategy  for  Changing  Loop  Parameters 

Each  node  will  normally  be  in  its  tracking  mode,  but  if  phase  error 
exceeds  <}>  then  the  node  shall  automatically  switch  to  its  fast  acquisition  mode 
and  remain  so  for  a period  of  time  T sufficient  to  reduce  the  frequency  error 
to  less  than  or  equal  to  1%  of  the  initial  frequency  offset.  After  this  elapsed 
period  of  time,  T the  node  will  automatically  switch  back  to  its  normal 
tracking  mode. 

As  already  observed,  the  phase  error  in  the  normal  tracking  mode  due  to  normal 
link  perturbations  or  oscillator  drifts  should  be  no  greater  than  1.3  ps.  Thus,  a threshold  of  / ^ 

= 2 ps  is  adequate  for  switching  from  normal  tracking  to  fast  acquisition  mode.  In  this 
case,  the  initial  amplitude  of  the  transient  due  to  the  initial  phase  offset  in  equation  (A58) 
may  be  obtained  from  (A58)  as  follows: 
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I 

2 ? uunbQ  c ^ Tfa  s .01  x fif 

Ln(Z  ^ uj  • b /.  01  x fif) 

or  T i 2 2 

fa  fuj  (A59) 

' n 

with  P = . 707,  uy  = . 014  and  <t>  , = b =3 
n th  o 

Ln(384.  72) 

Tfa  * T0099  = 601.26  sec.  (A60) 


A3. 9 Comparison  Between  Linear  and  Non  Linear  Models 

In  order  to  get  a comparison  between  the  acquisition  time  of  the 
linear  model  and  that  of  the  non  linear  sinusoidal  model,  we  must  choose  the 
same  parameters  as  are  shown  in  Fig.  A3,  i.  e.  , we  must  choose  loop  para- 
meters to  coincide  with  those  required  at  the  capture  point  of  the  non  linear 
model.  Although,  as  discussed  in  A3. 8. 5,  the  linear  model  would  actually 
start  its  acquisition  process  with  a non  zero  initial  phase  error,  a better 
comparison  between  the  two  can  be  obtained  if  it  is  assumed  that  the  initial 
phase  error  is  zero.  In  this  comparison,  the  settling  time  for  the  linear  model 
will  be  taken  to  be  that  value  of  time  for  which  the  phase  error  has  settled  to 
less  than  .01  times  the  peak  error.  The  results  for  the  linear  model  are  shown  in 
Fig  A5.  From  this  figure  we  note  that  the  fastest  settling  time  occurs  at  5 = .707 
but  that  there  is  little  difference  in  this  parameter  for  5 between  .4  and  1.  Also, 
the  peak  phase  error  never  exceeds  about  60°.  This  corresponds  to  no  cycles 
slipped  in  the  non  linear  one  cycle  phase  detector  case.  On  the  other  hand,  from 
Fig  A3  the  non  linear  model  slips  cycles  for  P > 1.  5 with  about  15  cycles  being 
slipped  at  the  P = 6 point.  The  ratio  of  the  settling  time  of  the  linear  model  to 
the  frequency  acquisition  time  for  the  non  linear  model  varies  from  approximately 
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Fig-  A5  Response  of  &{  of  second  order  PLL  using  integral  plus 

proportional  filter  with  extended  range  linear  phase  detector. 
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8 at  ? - .4  to  5 at  F = 6.  However,  settling  time  includes  part  of  the  corres- 
ponding phase  acquisition  process  of  the  non  linear  model.  If  phase  acquisition 
time  of  T - 4/B]  is  added  to  T^f  the  ratio  tg/(T^f  + T^)  becomes  about 
1.4  at  f = .4  and  5 at  P = 6.  It  has  been  suggested  that  the  point  in  time,  t^, 
at  which  the  peak  phase  error  occurs  in  the  linear  model  is  more  closely 
analogous  to  frequency  acquisition  time,  T f,  of  the  non  linear  model  than  is 
settling  time,  t . The  reason  for  this  is  that  the  point  of  peak  phase  error  is  the 

9 

time  at  which  the  frequency  of  the  VCO  is  first  equal  to  the  reference  frequency 

following  the  step  change  in  reference.  This  is  easily  shown  by  substituting 

equations  (A37),  (A38),  and  (A39)  into  (A34),  (A35),  and  (A36)  respectively. 

The  values  of  t are  shown  plotted  in  Fig.  A5  • A normalized  plot  of  output 
m 

frequency  versus  yu^t  is  shown  in  Fig.  A6  . From  the  curves  plotted  in  this 

figure  we  see  that  the  above  argument  is  approximately  correct  for  large  F,  but 

less  correct  for  small  The  output  frequency  curves  rise  rather  rapidly 

toward  the  new  reference  frequency  line,  cross  the  line,  and  then  settle  at  a much 

slower  rate  to  the  new  frequency.  For  P ? 1 the  output  frequency  curve  crosses 

the  reference  line,  peaks  above  the  line  and  then  settles  asymptotically  to  the 

reference.  For  0 < F < 1,  the  peak  occurs  between  the  first  and  second  crossing 

but  crosses  repeatedly  thereafter  with  exponentional  damping.  From  these  curves 

we  conclude  that  a better  measure  of  frequency  acquisition  time  for  the  linear 

model  is  that  time,  t , required  to  bring  the  output  frequency  to  within  some 

a 

small  fraction  \ of  the  reference  change  A f-  If  X iB  I&r8e  f°r  aH  p under 
consideration,  then  the  rise  time  constants  determine  tg.  If  \ iB  small  for  all 
* under  consideration,  then  the  fall  time  constants  determine  t . However,  if  X 

7>  a 

is  email  for  some  values  of  f under  consideration  but  large  for  others,  then  a 

mixture  of  rise  and  fall  time  constants  determine  t over  the  range  of  P 's.  For 

a 

example,  if  \ = . 01  thpn  for  the  (m  , F,)  pairs  required  at  the  capture  point  of  the 

n _ g 

non  linear  one  cycle  phase  detector  with  - 5 x 10  x fQ  - .0772  Hz,  the  following 
results  are  obtained: 
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Fig.  A6.  Output  Frequency  V 
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These  results  show  that  for  P ; = 5 

the  peak 

overshoot  and  hence,  the  entire 

fall  time  is  within  the 

X tolerance 

chosen. 

However,  this  does  not  indicate 

any  fault  in  the  choice 

of  Figure  A7  shows  a plot  of  output  frequency 

with  an  initial  pha 

se  o f fse  t . 

Figure  A8  and  Figure  A9  show  output 

phase  with  initial  frequency  offset  and  phase  error  respectively 
for  the  linear  model. 
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; Fig.  A9.  Output  Phase'  With  Initial  Phase  Offset 
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APPENDIX  B 


SURVIVABILITY  ANALYSIS 


The  details  of  a surviviabi I ity  analysis  discussed  in  Paragraph  8.1  are  presented  in 
this  section.  The  purposes  of  the  analysis  were  twofold.  First,  the  advantages  obtained  by 
using  a disciplined  timing  technique  over  the  independent  clock  approach  is  shown.  Of  course, 
the  independent  clock  approach  has  an  average  time  between  slips  of 

Ts  = B/8  . (B.l) 

Secondly,  the  performance  improvement  of  using  an  adaptive  timing  distribution  network  over 
the  fixed  approach  is  demonstrated. 

The  performance  of  several  approaches  for  implementing  a disciplined  timing 
distribution  system  will  be  evaluated.  The  parameter  B/8  is  considered  to  be  one  of  the 
fundamental  design  parameter  for  each  of  these  approaches.  In  this  case  B is  the  buffer  size, 
and  the  clock  offset,  8,  refers  to  the  offset  between  any  two  nodal  clocks  that  will  occur  if 
they  lose  their  references.  Normally,  all  nodes  will  be  phase-locked  to  a common  reference. 
Slips  will  not  occur  in  this  case.  The  only  time  that  slips  will  occur  is  when  one  or  both  of 
the  nodes  loses  its  reference  and  has  to  run  "asynchronously.  " A good  disciplined  approach 
will  attempt  to  minimize  the  product  of  the  amount  of  time  any  node  must  run  asynchronously 
and  the  offset,  8,  that  will  occur  during  this  time. 

The  first  approach  to  be  considered  is  the  use  of  a fixed  timing  distribution 
network.  That  is,  even  though  each  node  may  have  several  links  over  which  it  is  communi- 
cating, it  will  only  have  one  link  from  which  it  may  derive  a reference.  If  that  reference  is 

the  node  must  run  asynchronously . This  approacmwil I have  the  poorest  performance  in 

minimizing  the  amount  of  time  each  node  mustjrun  asynchronously.  The  performance 
. j.  p<  ndent  on  topology  of  the  network.  Because  of  this(,  completely  general  results 
■ in^d.  However,  by  assuming  a specific  topology  that  can  be  analyzed,  one 
i*  ‘ at  indicated  the  important  characteristics  of  this  approach.  The  topology 

. • i ■ e starred  polygon"  with  connectivity  4.  This  topology  for  an  8 node 

■ • jure  B.l  . Node  1 is  the  master,  and  timing  is  distributed  via  the  routes 


8 


5 


Figure  B.I . Starred  Polygon  With  Connectivity  of  4 and  8 Nodes 

indicated  by  the  arrows.  Note  that  there  are  four  independent  timing  chains  that  distribute 
timing  (1-2-4;  1-3;  1-7-5;  1-8-6). 

For  convenience  in  analysis,  we  will  assume  that  the  number  of  nodes  is  a mul- 
tiple of  4,  i.e.,  m - 4k.  We  will  first  calculate  the  average  fraction  of  communicating 
node  pairs  that  are  communicating  asynchronously.  Note  from  the  manner  in  which  timing  is 
distributed  that  each  timing  chain  contains  only  odd-numbered  nodes  or  even-numbered  nodes 
(other  than  the  master).  Thus,  Node  i is  always  synchronous  with  Node  i-2  or  Node  i+2 
if  those  links  are  operational  since  they  are  in  the  same  timing  chain.  However,  Node  i can 
communicate  asynchronously  with  Node  i-1  if  the  link  from  i to  i-1  is  operational  and  there 
is  an  outage  in  the  timing  chains  from  Node  1 to  Node  i or  from  Node  1 to  Node  i-1  (assume 
i < 2k+l).  There  are  a total  of  i-1  links  on  these  two  chains.  Then  the  probability  that  Nodes 
i and  i-1  are  communicating  asynchronously  as  a function  of  the  probability  of  link  outage. 


Pr(i  to  i-1  asynchronous)  = ('“Pej  1 — (1 — Pe ) ' ^ (B  2) 

There  are  2m  links  and  of  these,  an  average  of  2m  (l_Pe)  is  operational. 

Since  the  network  is  symmetrical,  the  average  fraction  of  communicating  nodes 
pairs  that  are  asynchronous  may  be  calculated  by  summing  twice  the  contributions  over  half 
the  network  (i=3,  2k+l ) and  adding  in  the  contributions  from  the  links  connecting  the  two 
halves  (Link  2 to  4k  and  Link  2k  to  2k+2).  This  fraction 

A = sir  [pn-p)2k-P3-'2P2+p] 
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is  then  given  by 


F = P-P) 
ac  2m(l  - p) 


i = 3 


1 - (1  - P);  “ 1 
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The  example  just  analyzed  had  only  one  reference  per  node  with  no  alternate 
references  available.  We  can  improve  the  performance  significantly  by  using  alternate 
references.  The  simplest  case  is  to  have  one  fixed  alternate  reference  per  node.  Thus, 
if  the  link  supplying  the  primary  reference  fails,  then  the  timing  subsystem  will  use  the 
alternate  reference.  If  both  of  these  references  fail,  then  the  nodal  clock  must  run  asyn- 
chronously. Note  that  this  timing  distribution  network  cannot  adaptively  reorganize  to 
achieve  the  optimum  configuration.  The  "starred  polygon  topology"  will  again  be  used  in 
the  analysis. 


The  network  topology  shown  in  Figure  B.l  is  assumed.  The  primary  reference  for 
each  node  is  shown  in  this  figure,  i.e.,  for  right  half  of  the  network  Node  i obtains  its 
primary  reference  from  Node  i-2.  We  will  assume  that  Node  i obtains  its  alternate  reference 
from  Node  i-1  in  the  right  half  of  the  network  and  from  Node  i+1  in  the  left  half  of  the 
network. 


Again  since  the  network  is  symmetrical  we  will  treat  just  the  right  half.  When 
Nodes  i and  i-2  are  communicating  it  is  always  synchronously  since  this  link  is  the  first  choice 
of  Node  i with  which  to  derive  a reference.  Node  i communicates  asynchronously  with  Node 
i-1  only  when  Node  i is  deriving  its  reference  from  Node  i-2,  (i.e.,  both  Links  i to  i-1  and 
i to  i-2  are  operational)  and  Nodes  i-1  and  i-2  are  asynchronous.  Proceeding  back  toward 
the  master  we  find  that  Nodes  i-1  and  i-2  are  asynchronous  if  both  Links  i-1  to  i-3  and  i-1 
to  i-2  are  down  or  if  Link  i-1  to  i-3  is  operational  and  Nodes  i-2  and  i-3  are  asynchronous, 
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etc.  Then  for  large  m (ignoring  end  effects  ) the  probability  that  Nodes  i and  i-1  are  com- 
municating asynchronously  is 


P.  = Pr  (i  to  i-1  asynchronous) 


(1  -p2)  [ p2  + (1-p)  [p2  + (1-p)  p' 


■=  p2  <i-p)2  E o-p)k 
k=0 


= p(l-p)' 


1 - (7-p)1"2 


(B.4) 


Then  as  before  the  fraction  of  nodes  communicating  asynchronously  is  obtained 
by  summing  over  all  links  and  is  given  by 

2k+l 


ac 


2m(l-p)  i=3 


0 -p) 
4k 


. ,3  . 2k+2 

(2k-l)p-(l-p)  + (1-p) 


(B.5) 


If  Equation  (B.5)  is  expanded  in  powers  of  p we  find  that  the  leading  term  is 
o 

p . This  is  consistent  with  the  approach  of  having  two  available 
references  per  node.  Then  two  link  outages  can  cause  a node  to  lose  its  reference  to  the 
master  and  become  asynchronous  with  respect  to  the  rest  of  the  network.  This  term  gives 
the  asymptotic  behavior  of  FQC  for  small  p.  In  addition,  for  small  p and  large  k,  Fac%  p2/2. 

The  fraction  of  communicating  node  pairs  that  are  asynchronous  is  plotted  in 
Figure  B.2  as  a function  of  p,  the  link  outage  probability  for  networks  with  20  and  80 
nodes  (m  = 20  and  80).  As  one  would  expect,  the  performance  degrades  as  the  number 
of  nodes  in  the  network  increases  because  the  connectivity  of  the  network  is  fixed  (equal 
to  4)  and  thus  the  length  of  the  timing  chains  will  grow  giving  a higher  probability  that 
any  chain  will  be  disrupted  due  to  link  failures.  This  effect  is  more  pronounced  in  the  case 
where  no  alternate  reference  is  available.  However,  the  worst-case  values  of  F are  not 
to  different  for  m = 20  and  m - 80. 
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Figure  B~2.  Fraction  of  Communications  Nodes  that  are 
Asynchronous  as  a Function  of  Probability  of  Link  Outage 
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These  results  can  be  related  to  slip  rate  by  observing  that  the  only  time  a node 
experiences  a slip  is  when  it  is  communicating  asynchronously  with  another  node.  In  this 
case  the  time  between  slips  is  B/8.  Then  the  average  time  between  slips  at  each  node  in  the 
network  may  be  estimated  as 

Ts  = B/8  Fac  (B.6) 

This  quantity  is  plotted  in  Figure  B.3  for  m = 20.  Note  that  the  performance  is  better  than 
the  independent  clock  technique  for  the  same  value  of  B/8  at  all  vaules  of  p.  We  also  note 
that  if  one  alternate  reference  is  used,  the  value  of  B/8  can  be  reduced  to  one-fifth  that  of 
the  independent  clock  technique  and  still  maintain  better  performance  over  all  values  of  p. 

Although  these  results  indicate  that  some  performance  improvements  may  be 
obtained  by  using  one  or  more  fixed  references  in  a master-slave  manner,  these  improvments 
are  not  that  great  when  the  link  outage  probability  is  high  because  the  timing  distribution 
network  is  disrupted  so  frequently.  This  is  the  reason  that  master-slave  systems  have  been 
frequently  criticized  because  of  poor  survivability.  However,  one  can  make  an  additional 
improvement  in  a master-slave  system  by  giving  it  the  capability  of  adaptively  reconfiguring 
the  timing  distribution  network  as  link  outages  occur.  This  approach  has  the  properties  that: 

1 . All  communicating  nodes  are  synchronous  in  the  steady  state  (after  a suffi- 
ciently long  period  following  the  last  failure). 

2.  The  only  time  a nodal  clock  will  act  as  a self-reference  is  while  the  timing 
distribution  network  is  being  reconfigured  (which  is  a relatively  short 
period  of  time). 

This  approach  has  a very  high  degree  of  survivability.  The  system  can  be  designed  so  that 
no  slips  occur  in  the  steady  state.  The  only  time  slips  can  occur  is  during  the  transients 
associated  with  reconfiguring  the  network.  Obviously,  one  of  these  transients  occurs  every 
time  an  outage  occurs.  In  the  following  analysis  the  average  time  between  slips  will  be 
calculated  as  a function  of  the  rate  at  which  link  outages  are  occurring. 
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Figure  B~3.  Average  Time  Between  Slips  as  a Function  of 
the  Probability  of  Link  Outage  for  m = 20 
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Let  T represent  the  time  required  to  reconfigure  the  timing  network  (a  function 
of  topology)  and  T * represent  the  time  required  to  bring  the  nodal  clocks  back  into  lock 
after  the  timing  network  is  reestablished.  The  average  length  of  time  a link  is  operational 
before  a failure  is  denoted  by  1/ X (second),  and  the  average  link  outage  time  is  denoted  by 
1/p  (second).  Then  the  probability  of  a link  outage  is 

2 

P A+F  (B.7) 

Each  node  needs  a reference,  a significant  fraction  of  the  total  time  while  the  network  is 
under  stress  if  slips  are  to  be  avoided.  If  the  nodal  clock  can  act  as  a self-reference  for  at 
most  T seconds  (T  = B/8)  without  a slip  (or  buffer  overflow ),  then  within  T seconds  of  a 
failure,  the  nodal  clock  needs  to  acquire  a reference  for  long  enough  (r*  seconds)  to  allow 
resynchronization.  Thus,  after  a failure  a total  of  x = T+  T*  seconds  with  no  failures  is 
required  to  allowall  nodes  affected  by  the  failure  to  select  a new  reference  and  resynchronize. 
The  x failure-free  seconds  must  occur  within  T seconds  of  the  failure  or  slips  will  occur. 

It  would  be  desirable  to  analytically  determine  the  average  time  between  slips  as  a function 
of  T,  x,  \ , and  p,  but  this  is  a very  unwieldy  analytical  problem.  However,  we  can  make 
some  rough  estimates  that  will  illustrate  the  important  trade-offs. 

To  simplify  the  analysis  the  link  up  times  will  be  considered  to  be  exponentially 
distributed,  i.e.. 


p (t)  = X e ^ f . 

Then  the  probability  that  any  specific  link  fails  during  the  interval  (0,  x)  is 
Pr  (link  failure)  = p^ 


(B  .8) 


(B.9) 
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The  probability  that  there  are  no  link  failures  in  k operational  link  in  (0,  x)  is 

k 

Pr  (no  failures  in  k links)  = (1-Pf) 

-k  \ x 


(B.10) 


Finally,  the  probability  that  there  are  one  or  more  failures  in  k links  is 

“ k A x 

Pr  (one  or  more  failures)  = 1 - e 


(B.ll) 


Wewillnow  proceed  through  a relatively  crude  analysis  to  demonstrate  that 
with  an  adaptive  timing  distribution  network  approach  very  high  failure  rates  must  be  experi- 
enced before  there  is  any  chance  of  slips.  A specific  network  topology  is  not  used.  We  will 
only  make  assumptions  about  the  length  of  the  timing  chains.  It  will  be  assumed  in  this  analysis 
that  the  timing  chain  between  any  node  and  its  master  has  k links.  Then  Equation  (B.ll)  gives 
the  probability  that  a failure  occurs  in  the  timing  chain  during  the  interval  (0,  x)  when  the  node 
is  attempting  resynchronization.  To  obtain  a conservative  estimate  of  the  probability  of  not 
achieving  resynchronization,  we  divide  the  interval  (o,  T)  into  T/x  intervals  each  of  length 
x | Lx  | represents  the  integer  part  of  (x).  Each  of  these  J T/x|  intervals  provides  an  opportunity 
to  resynchronize  in.  Then  the  probability  that  resynchronization  cannot  be  achieved  is  equal 
to  the  probability  that  there  are  link  failures  in  each  of  these  | T/x  | intervals,  i.e., 


Rp  (no  resync) 


= (l  -e"kAx)lT/x  I 


(B.  1 2) 


If  resynchronization  is  not  achieved  within  T seconds,  a slip  is  assumed  to  occur.  Then  the 
nodal  clock  has  another  T seconds  within  which  to  resynchronize  or  a second  slip  will  occur, 
etc.  The  average  number  of  slips  that  occur  before  resynchronization  is  achieved  is 


N = 
s 


+ 3 P 


O-Pnrs)' 


(B.13) 


* 


The  average  time  between  failures  in  a timing  distribution  chain  with  k links 


Tf  k X N 

s 


(B.14) 


where  is  given  by  Equation  (B.13). 

The  average  time  between  slips  as  a function  of  the  number  of  failures  per  day 
(Fd  = 86,400  A)  per  link  is  shown  in  Figure  B.4  for  several  values  of  the  parameters.  The  number 
of  links  between  any  node  and  its  reference  is  set  a k = 1 0 . Performance  is  improved  by  decreasing 
x,  the  failure-free  interval  required  for  resynchronization,  and  by  increasing  B/5  , the  maximum 
interval  over  which  a node  may  act  as  a self-reference  without  slips.  The  values  used  for  the 
parameters  x and  B/8  in  this  figure  are  in  the  range  that  would  probably  be  implemented.  Note 
that  the  average  time  between  slips  is  extremely  long  even  for  very  high  failure  rates.  One 
would  probably  expect  at  most  just  a few  failures  per  day  per  link  even  under  severe  stress 
because  a high  level  of  stress  would  imply  greater  difficulty  in  making  repairs.  Certainly, 
there  could  be  many  failures  caused  by  physical  damage  which  could  fake  days  or  longer  to 
repair.  Thus,  the  nature  of  most  failures  is  not  such  that  a link  may  fail,  be  repaired  in  o few 
minutes,  then  fail  again  a short  time  later  followed  by  a quick  repair,  etc.  Certainly  this 
type  of  failure  mechanism  which  could  lead  to  a higher  value  of  Fj  poses  the  only  threat  to 
a master-slave  timing  subsystem  with  adaptive  reorganization.  The  reason  for  this  is  that  links 
which  behave  in  this  way  cause  the  network  to  have  to  reorganize  too  often.  Such  behavior 
could  be  caused  by  a link  which  was  frequently  jammed  for  short  periods  of  time.  Thus,  an 
enemy  could  fry  to  disrupt  the  synchronization  subsystem  by  frequently  jamming  a number  of 
links.  However,  this  tactic  can  be  effectively  countered  by  not  allowing  any  node  to  use 
a link  as  a reference  which  has  failed  more  than  a few  times  a day. 

These  results  indicate  that  the  performance  of  master-slave  timing  subsystem  with 
adaptive  reorganization  is  greatly  superior  than  that  of  an  independent  clock  timing  subsystem 
at  virtually  all  stress  levels  even  using  much  smaller  values  of  B/8  . One  might  add  that  pre- 
cisely the  same  comments  could  be  made  about  the  time  reference  distribution  approach  because 
it  also  uses  the  adaptive  reorganization  technique.  This  is  the  key  to  achieving  a high  level 
of  performance  under  stress. 
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The  mutual  synchronization  approach  also  achieves  a high  degree  of  survivability. 
In  fact,  one  might  even  think  that  it  should  perform  better  unde>  stress  than  the  adaptive 
reorganization  approach  since,  if  a node  has  at  least  one  operational  link  it  has  a reference. 
Thus,  in  a pure  mutual  sync  network  nodes  never  communicate  asynchronously.  In  actuality, 
though,  the  master-slave  system  with  adaptive  reorganization  can  be  designed  so  that  even 
under  severe  stress,  the  probability  that  node  pairs  have  to  communicate  asynchronously  for 
long  enough  to  result  in  slips  (or  buffer  overflows)  is  extremely  small.  Thus, the  mutual  sync 
approach  would  offer  no  measurable  advantage.  In  fact,  it  is  likely  that  there  will  be  more 
variation  in  nodal  clock  frequencies  (or  more  error  propagation)  with  the  mutual  sync  approach 
under  stress  which  is  an  undesirable  characteristic. 
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APPENDIX  C 


IMPACT  OF  FUTURE  TECHNOLOGY 


An  activity  that  was  addressed  during  the  course  of  this  study  was  an  assessment 
of  future  technology  and  its  impact  on  the  selection  of  the  most  appropriate  timing  subsystem. 

It  was  concluded  that  several  areas  of  advancing  technology  would  affect  capabilities  of  timing 
subsystems  but  that  no  one  timing  technique  would  be  more  substantially  affected  than  any  other 
timing  technique.  It  was  therefore  concluded  that  the  impact  of  future  technology  would  not 
influence  the  evaluations  of  either  timing  techniques  or  timing  subsystems. 

The  particular  areas  investigated  are  discussed  in  the  paragraphs  below: 


Clocks 

In  the  area  of  stable  clocks  the  specific  point  being  investigated  was  the  possibility 
of  a price  breakthrough  that  would  essentially  make  the  independent  clocks  approach  more  cost 
competitive  with  the  disciplined  approaches. 


Clock  possibilities  range  between  quartz  crystal  at  one  extreme  through  rubidium 

gas  cells,  cesium  beam  devices  to  hydrogen  masers.  These  clocks  at  this  time  provide  daily 

stabilities  ranging  between  1 part  in  10^  through  1 part  in  10^.  The  potential  improvement 

in  these  devices  is  at  least  one  order  of  magnitude  better  daily  accuracy,  but  this  is  yet  to 

come.  There  are  several  new  concepts  that  could  be  available  within  the  next  10  yeai..  These 

include  the  techniques  known  as  the  dual  crystal,  the  cesium  gas  cell,  the  passive  hydrogen 

standard,  a utility  atomic  standard  and  a superconducting  cavity  oscillator.  These  techniques 

have  predicted  daily  stabilities  that  range  between  1 part  in  10^'  to  1 part  in  lo'^.  The 

general  evolution  in  cost  of  all  devices  is  toward  cheaper  implementations,  but  the  cesium  and 

hydrogen  approaches  are  still  predicted  to  be  rather  expensive.  A change  of,  say,  $100,000 

to  $50,000  for  hydrogen  devices  is  a dramatic  reduction,  but  still  leaves  a lot  to  be  desired  from 

a single-item  cost  standpoint.  The  short-term  stabilities  of  most  of  these  devices  are  roughly 
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around  the  same  general  order  of  1 part  in  10  through  1 part  10  even  considering  predictions 
to  the  future.  No  dramatic  breakthrough  is  expected  in  this  area. 
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Integrated  Circuitry 

Current  technology  in  large-scale  integrated  circuitry  is  available  in  both  standard 
off-the-shelf  and  custom  computer-aided  designed  varieties  in  a number  of  technologies.  In  the 

12  3 

high-speed  area,  these  include  the  ECL,  IIL,  and  CCD  techniques.  Lower  speed  technologies 
that  are  currently  available  are  metalgate  CMOS,  PMOS  and  SOS  CMOS.  The  trends  are  to 
provide  possibly  larger  chips  with  greater  density  of  individual  elements  and  possibly  an  evolving 
multilevel  interconnection  capability  that  is  producible.  Effort  will  continue  throughout  industry 
to  reduce  the  cost  such  that  competition  will  drive  the  per  gate  cost  down  on  logic -oriented  LSI 
devices.  Computer-aided  design,  which  is  used  in  many  organizations  throughout  industry, 
serves  to  optimize  the  total  process  of  design,  fabrication,  test,  and  implementation.  Hybrid 
multilayer  thin-film  substrates  are  currently  used  for  extremely  small,  reliable  and  cost-effective 
equipments.  Evolution  in  this  area  would  be  towards  a higher  density  of  large-scale  integrated 
circuits  implemented  on  hybrids  with  a continuation  of  individual  chip  packaging  techniques 
which  foster  increased  yield  and  reliability  of  the  end  product. 

The  advances  in  this  area  while  most  pertinent  to  the  Time  Reference  Distribution 
approach  did  not  appear  to  significantly  impact  either  the  approach  or  its  evaluation. 

Microprocessors 

There  is  a current  proliferation  of  microprocessors  available  as  off-the-shelf  devices 
which  already  have  basic  programs  developed.  These  are  available  in  several  technologies 
including  Schottky-TTL,  PMOS,  NMOS  and  very  shortly,  SOS  CMOS.  The  microprocessor  will 
become  a circuit  element  used  throughout  systems  as  one  might  use  a transistor  or  resistor  as  time 
goes  on.  The  evolution  in  technology  will  probably  see  the  same  size  chips  being  used  but  the 
number  of  components  per  chip  increasing  dramatically.  The  IIL  technology  has  the  possibility 
of  10,000-50,000  components  per  chip  depending  upon  fabrication  advances.  That  is,  is  it  not 
possible  to  economically  build  chips  like  this  today  but  that  limitation  could  go  away  with  time? 
There  would  be  some  increase  in  speed  but  there  is  plenty  of  speed  capability  in  the  current 
repertoire  of  technology  available  for  microprocessors.  IIL  should  allow  operation  of  100  MHz  or 
more.  As  far  as  microprocessor  architecture  development  goes,  more  CPU  functions  will  likely 


^Emitter  Coupled  Logic 
^Integrated  Injection  Logic 
^Charge  Coupled  Devices 


be  placed  on  an  individual  chip  and  functions  such  as  multiply  divide  will  be  integrated  into  the 
fundamental  CPU  chip.  Many  applications  now  tolerate  a slow-speed  microprocessor  as  long  as 
a high-speed  multiply/divide  function  is  separately  associated  with  said  microprocessor.  The 
fundamental  problem  of  pin  limitation  will  persist.  A breakthrough  of  some  sort  is  solely  required 
in  this  area  in  that  it  is  just  not  possible  to  physically  make  many  more  connections  to  an 
individual  chip.  Associated  with  the  basic  microprocessor  are  memories  such  as  PROM,  EAROM, 
RAM,  and  ROM  devices  where  LSI  application  could  derive  a megabit  storage  capability.  In  the 
area  of  Charge-Coupled  Devices  (CCD),  it  has  been  estimated  that  in  the  not  too  distant  future, 
as  many  as  two  million  bits  per  chip  of  storage  can  be  available.  Some  guesses  are  much  higher. 
Bubble  memories,  although  not  heavily  in  use  at  this  time,  could  afford  an  area  of  breakthroughs 
to  occur. 

The  microprocessor  also  provides  the  opportunity  to  design  complex  self-healing  or 
checking  subsystems  that  could  be  integrated  into  a highly  reliable  timing  subsystem  or,  indeed, 
any  subsystem.  Other  possibilities  are  precise  clock  correction  through  extremely  detailed  and 
complex  predictive  techniques  which  could  be  implemented  through  complex  hardware  in  a cheap 
and  efficient  manner  using  the  microprocessor  as  the  fundamental  element. 

While  microprocessor  technology  has  a direct  impact  on  the  implementation  of  the 
Time  Reference  Distribution  approach  current  capabilities  and  costs  already  guarantee  cost- 
effective  implementation  of  timing  subsystem  functions.  Future  advances  while  interesting  and 
applicable  will  be  of  extremely  marginal  importance. 

Optical  Circuitry 

The  optical  circuits  area  is  rampant  with  new  developments  and  innovations.  Fiber 
optic  techniques  are  now  being  considered  for  application  to  long-haul  communication  systems, 
secure  protected  communication  links,  ground-to-bal loon  communication  links,  instrumentation 
for  equipments  in  an  EMP  environment  or  other  hardened  requirement,  TEMPEST  design  approaches 
for  RED/BLACK  applications  and  internal  wiring  interconnections  which  allow  fail-safe  crypto- 
graphic design  and  EMC  acceptable  designs.  The  advent  of  an  optical  switch  or  computer  could 
dramatically  change  the  timing  needs  associated  with  a nodal  switch.  The  multiplex  scheme  in 
an  optical  environment  could  require  dramatically  different  timing  accuracies  and  organization. 
Optical  storage  media  are  currently  available  in  terms  of  film  and  holographic  devices. 
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The  probability  of  transmitting  significantly  higher  bit  rates  puts  further  emphasis 
on  the  need  for  accuracy  in  the  timing  subsystem.  This  point  has  been  considered  throughout 
the  study . 
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